Fix slow and flaky Android tests in CI
Diagnose timing, state and device causes of flaky Appium runs, and use logs, recordings, profiles and run history to make failures reproducible.
Most flaky Android tests are not random. They are timing assumptions, shared state, or a device profile that changed under the suite. The fix starts with evidence, not retries.
Separate the failure classes
| Symptom | Likely cause | First check |
|---|---|---|
| Passes locally, fails in CI | Timing or a slower cold boot | The run's first-frame time and boot_ms. |
| Passes on retry | Async work not awaited | The failing step, then the app's logcat around it. |
| Fails only in parallel | Shared backend or app data | Whether shards share an account or tenant. |
| Fails on one Android version | Framework or permission behavior | The session's device_profile. |
| Blank screen, test then fails | Stream or app launch, not the test | The screen recording and the launch stage. |
Make waits explicit
Use the client's wait primitives, not sleep. A fixed sleep that is long
enough on your laptop can still be too short on a cold device.
from appium.webdriver.common.appiumby import AppiumBy
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 20)
wait.until(
EC.element_to_be_clickable((AppiumBy.ACCESSIBILITY_ID, "Add to cart"))
).click()
wait.until(
EC.visibility_of_element_located(
(AppiumBy.ID, "com.acme.checkout:id/total")
)
)Disable animations for the test build, or grant the animation scale through the app's test configuration. UiAutomator2's idle waits cannot settle a spinner that never stops.
Reset state between tests
Each SpawnDroid session starts on a clean device, but a suite that runs several tests in one session shares app data. Either:
- clear the app between tests with the UiAutomator2
mobile: clearAppcommand, or - start one session per test file so each file gets a clean install.
The first is faster; the second is easier to reason about when a test writes server-side data.
Use the evidence the run already keeps
- Logcat: the live stream while the session runs; the final capture on
Personal and above. Search for the app's package,
FATAL EXCEPTION,ANRand the timestamp of the failing step. - Screen recording: retained on Personal and above. Watch the last 20 seconds before the failure; a missed tap is visible even when the stack trace is not.
- Run history: the run page records the outcome, the device profile, the duration and the failure code. A failure that clusters on one profile is a device/version issue, not a test issue.
- Since last green: a failed run compares against the last passing run of the same app, including whether the same commit ever passed. That separates a regression from a flake.
Retry with a bound
One retry is useful for a known transient. A retry loop that hides a failure is not. If you retry, record the first failure: SpawnDroid keeps the run's result and artifacts, and the retry is a separate run. Alert when the pass-on-retry rate rises; it is an early signal that waits or state are wrong.
Pin the device profile
A suite that silently moves between Android versions is a flaky suite by
construction. Pass profileId when starting the runner, for example
android-14-phone, and change it deliberately. The profile is recorded on
every run.
Run it on SpawnDroid
Every run keeps the device profile, outcome and failure code; Personal and above add the retained logcat and screen recording. Start free with 200 device-minutes, then open the run history in the console to compare.
Was this page helpful?
Run Android tests in parallel
Shard an Appium or WebdriverIO suite across parallel SpawnDroid runners, size the job matrix to the plan, and calculate the device-minute cost.
Test an APK without Android Studio
Upload an APK or AAB, run it on a live Android device in the browser, and share it with someone else. No SDK, emulator or USB cable.