CI / GitHub Actions

Token

A CI runner cannot open a browser to sign in, so it signs in with a personal token. Create one on your own machine — the browser opens once for SSO, and the token is printed once, never again:

tapstep token create --name github                  # prints tapstep_pat_… on stdout
tapstep token create --name github | gh secret set TAPSTEP_TOKEN   # straight into a repo secret
tapstep token create --name nightly --expires-days 90

Only the token goes to stdout, so it pipes cleanly; the note about it goes to stderr. The desktop app does the same under Settings → General → CI tokens: create, copy once, revoke.

Put the token in GitHub → your repository → Settings → Secrets and variables → Actions as TAPSTEP_TOKEN, and expose it to the job as an environment variable — TAPSTEP_TOKEN in the environment counts as signed in.

tapstep token list                # id, name, prefix, created, last used, expires — never the token
tapstep token revoke github       # by id, or by name when the name is unique

How the CLI checks it: a tapstep_pat_… token is verified against api.tapstep.dev once and the answer is cached in ~/.tapstep/token-check.json for 24 hours — on a fresh runner that is one request per job. A revoked, expired or unknown token stops the command with an error. If the API cannot be reached, the run goes on — with a warning: line on stderr when this machine has never verified the token (a fresh runner), silently when it has. An outage on our side never turns your pipeline red.

A TAPSTEP_TOKEN set to anything else — older docs let any string through — still works, with a warning on stderr, until 0.15. Replace it with a token from tapstep token create before then.

Minimal workflow (web)

.github/workflows/e2e.yml:

name: e2e
on: [push, pull_request]

jobs:
  web:
    runs-on: ubuntu-latest
    env:
      TAPSTEP_TOKEN: ${{ secrets.TAPSTEP_TOKEN }}
      TAPSTEP_NO_SANDBOX: "1"   # Ubuntu 24.04 blocks Chromium's sandbox, see below
    steps:
      - uses: actions/checkout@v4

      - name: Install tapstep
        run: |
          curl -fsSL https://s3.tapstep.dev/tapstep-cli/tapstep-latest-linux-x86_64.tar.gz | tar -xz
          sudo mv tapstep /usr/local/bin/

      - name: Validate flows
        run: tapstep validate flows/

      - name: Run flows
        run: tapstep test flows/ --report junit --output results.xml

      - name: Upload results
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: tapstep-results
          path: |
            results.xml
            reports/
          if-no-files-found: ignore

tapstep validate parse-checks every flow and its includes without a device, so a typo fails the job in a second instead of halfway through the suite.

The exit code tells a red suite from a broken runner: 1 — flows ran and some failed; 2 — tapstep could not run them (bad flag, no device, unparseable config). Both fail the job; only 1 means the product is red.

The GitHub Ubuntu image already has Google Chrome, and tapstep picks it up. --browser managed downloads a pinned Chromium into ~/.tapstep/browsers instead, so CI and every laptop run the same browser version — cache that folder between jobs if you use it.

TAPSTEP_NO_SANDBOX=1 is there because Chromium’s sandbox does not start on ubuntu-latest (Ubuntu 24.04): AppArmor restricts unprivileged user namespaces, and Chromium exits with No usable sandbox. In container: jobs and most Docker images Chromium runs as root and refuses the sandbox outright (Running as root without --no-sandbox is not supported). Since 0.13.12 tapstep spots either failure on Linux, prints a notice and relaunches Chromium once with --no-sandbox, so a job without the variable still runs — it just pays for a failed first launch. TAPSTEP_NO_SANDBOX=1 skips that attempt and starts without the sandbox right away.

Reaching your app

Three ways, depending on where the app under test lives.

A staging URL. Keep the address out of the flow and pick it per environment in config.yaml:

config.yaml

environments:
  local: { BASE_URL: http://localhost:3000 }
  ci:    { BASE_URL: https://staging.example.com }

flows/login.flow.yaml

appId: ${BASE_URL}/login

      - name: Run flows
        run: tapstep test flows/ --env-set ci --report junit --output results.xml
        env:
          APP_PASSWORD: ${{ secrets.APP_PASSWORD }}

--env-set ci loads environments.ci and a local .env.ci if there is one; ${TAPSTEP_ENV} is ci inside the run. Secrets do not need a file: any ${NAME} no layer sets falls back to the process environment, so a GitHub secret exported in env: reaches ${APP_PASSWORD} as is. See config.yaml.

The app in the same job. Start it in the background, wait for the port, then run:

      - run: npm ci && npm run build
      - run: npm start &
      - run: curl -fsS --retry 30 --retry-delay 2 --retry-connrefused http://localhost:3000 > /dev/null
      - run: tapstep test flows/ -e BASE_URL=http://localhost:3000 --report junit --output results.xml

Or write appId: http://localhost:3000 in the flow directly.

No UI at all. device: none flows made of request: and sql: steps need no browser or device and run on any runner — see API testing.

Android

Boot an emulator with reactivecircus/android-emulator-runner and run tapstep inside it. tapstep finds the running emulator through adb on its own:

  android:
    runs-on: ubuntu-latest
    env:
      TAPSTEP_TOKEN: ${{ secrets.TAPSTEP_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - name: Enable KVM
        run: |
          echo 'KERNEL=="kvm", GROUP="kvm", MODE="0666", OPTIONS+="static_node=kvm"' | sudo tee /etc/udev/rules.d/99-kvm4all.rules
          sudo udevadm control --reload-rules
          sudo udevadm trigger --name-match=kvm
      - name: Install tapstep
        run: |
          curl -fsSL https://s3.tapstep.dev/tapstep-cli/tapstep-latest-linux-x86_64.tar.gz | tar -xz
          sudo mv tapstep /usr/local/bin/
      - uses: reactivecircus/android-emulator-runner@v2
        with:
          api-level: 34
          arch: x86_64
          disable-animations: true
          script: |
            adb install -r app/build/outputs/apk/debug/app-debug.apk
            tapstep test flows/android/ --report junit --output results.xml

Each line of script: runs as its own command. With several devices attached, --device <serial> picks one.

iOS

iOS flows do not run in CI yet: the release binary does not ship the XCUITest runner the iOS driver needs. Run them on a Mac where the runner is installed; CI support comes with a release that ships it.

VS Code extensions

A vscode: flow boots the installed desktop VS Code, so the job installs it and gives it a virtual display:

  vscode:
    runs-on: ubuntu-latest
    env:
      TAPSTEP_TOKEN: ${{ secrets.TAPSTEP_TOKEN }}
    steps:
      - uses: actions/checkout@v4
      - name: Install VS Code and Xvfb
        run: |
          curl -fsSL -o code.deb "https://update.code.visualstudio.com/latest/linux-deb-x64/stable"
          sudo apt-get update && sudo apt-get install -y ./code.deb xvfb
      - name: Install tapstep
        run: |
          curl -fsSL https://s3.tapstep.dev/tapstep-cli/tapstep-latest-linux-x86_64.tar.gz | tar -xz
          sudo mv tapstep /usr/local/bin/
      - run: npm ci && npx @vscode/vsce package -o dist/my-ext.vsix
      - run: xvfb-run -a tapstep test flows/vscode/ -e VSIX=${{ github.workspace }}/dist/my-ext.vsix --report junit --output results.xml

The .vsix comes from the build step before it; a flow reads it in its vscode: header as vscode: { vsix: ${VSIX} }, or names a testspace: from config.yaml. A relative vsix: resolves against the flow’s own folder (a testspace’s against the project root), so pass an absolute path as above — see config.yaml. tapstep finds VS Code through the code CLI on PATH, or TAPSTEP_VSCODE=/path/to/code.

Windows runners

On runs-on: windows-latest, install from the zip in PowerShell:

      - name: Install tapstep
        shell: pwsh
        run: |
          Invoke-WebRequest https://s3.tapstep.dev/tapstep-cli/tapstep-latest-windows-x86_64.zip -OutFile tapstep.zip
          Expand-Archive tapstep.zip -DestinationPath "$env:LOCALAPPDATA\tapstep\bin"
          "$env:LOCALAPPDATA\tapstep\bin" | Out-File -Append -Encoding utf8 $env:GITHUB_PATH

Reports

--report junit|allure|html|json with --output <file> writes one combined report for a directory run; allure writes an allure-results/ directory (in the current folder, or --output <dir>). A report: block in config.yaml writes a report on every run without any flag. Failure screenshots and network logs land under reports/ — upload that folder with if: always(), as in the workflow above. See Reports.

Flows with AI steps need a provider in CI too (TAPSTEP_AI_MODEL, TAPSTEP_AI_KEY and TAPSTEP_AI_ENDPOINT from secrets, or an ai: block in config.yaml). With privacy mode on, run tapstep privacy install while building the runner image — the masking model is a ~945 MB download you do not want in every job.