Run Android tests in parallel

Parallel Android tests are limited by two numbers: the plan's parallel runners and the total device-minutes. Sharding the suite across runners cuts wall-clock time; it does not reduce device-minutes, because each runner holds its own device for the whole shard.

How SpawnDroid models itCopy Anchor Link

• One runner is one live Android device.
• Hobby has 1 runner, Personal 2, Team 5, Enterprise 25+.
• A start call either gets a runner or returns HTTP 429 with code: "concurrency". There is no queue.
• Device-minutes are counted per runner from ready to teardown. Two runners for five minutes consume ten device-minutes.

Shard with a matrixCopy Anchor Link

Use your runner's shard support — Jest has --shard, pytest has pytest-split, and WebdriverIO can take an explicit spec list. Keep one session per matrix job:

name: Android UI tests
on: pull_request
jobs:
  android:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3, 4]
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 22, cache: pnpm }
      - run: pnpm install --frozen-lockfile
      - name: Run shard on a hosted device
        uses: kl3jvi/spawndroid/actions/android-test@v1
        with:
          api-key: ${{ secrets.SPAWNDROID_API_KEY }}
          app-path: app/build/outputs/apk/debug/app-debug.apk
          command: pytest --splits 4 --group ${{ matrix.shard }} --durations-path .test_durations

Four jobs need four parallel runners. On Team (5 runners) the matrix fits; on Personal (2) only two shards run at once, and the other two are refused with 429. Size the matrix to the plan or keep a smaller matrix and accept a longer wall-clock time.

Split by test file, not by testCopy Anchor Link

Sharding by individual test can put one long test in a shard that finishes late. Prefer a deterministic split by spec file or by a stable hash of the test name. pytest-split does this for pytest:

pip install pytest-split
pytest --splits 4 --group 1 --durations-path .test_durations

Generate .test_durations once from a previous run and commit it. The split then stays stable as tests are added.

Cost exampleCopy Anchor Link

A suite that takes 40 minutes serially, with 4 shards, runs in about 10 minutes of wall clock but still consumes about 40 device-minutes. On Hobby that is 20% of the monthly allowance; on Team it is about 1%. The saving is pipeline time, not device time.

When not to parallelizeCopy Anchor Link

• A suite with shared backend state can race itself. Give each shard its own test account or tenant.
• A single long test is not helped by more runners.
• The CI job's own setup (build, install, npm) can dominate a short suite.

Run it on SpawnDroidCopy Anchor Link

Personal includes 2 runners, Team 5, and Enterprise 25+. Compare plans or start free with one runner and 200 device-minutes.