Mobile locators and gestures

Finding elements on a phone screen, and interacting with them, works differently from the web — there is no CSS, and the interactions include taps, swipes and scrolls that a mouse never had. Getting locators right on mobile is, as on the web, the difference between a robust suite and a brittle one; and gestures are a whole category of action you must handle. This lesson is mobile locators and gestures, carrying over the "prefer user-facing, stable locators" principle to native apps.

Locating elements on a native screen

A native app has no DOM of HTML elements; it has a native element tree (Android's View hierarchy, iOS's view hierarchy). Appium exposes it, and you locate elements with strategies analogous to the web's — but the best choice, again, is the user-facing, stable one:

• Accessibility id (~name) — the recommended locator. It maps to the element's accessibility label (Android's content-desc, iOS's accessibilityIdentifier). It is stable, cross-platform (the same accessibility id can work on both Android and iOS if developers set it), and — like getByRole on the web — doubles as an accessibility check. driver.$('~sign-in-button').
• Id / resource-id — the platform's element id (Android resource-id, iOS name). Stable but platform-specific.
• Text / label — locate by visible text (driver.$('android=new UiSelector().text("Sign in")') or an XPath on text). Readable but breaks with translations and wording changes.
• XPath over the native tree — powerful but slow and brittle on mobile (the tree can be large and queries are expensive). Use it as a last resort, exactly as CSS/XPath is a last resort on the web.

The guidance mirrors the web-locators lesson: prefer accessibility ids, which are user-facing and stable and encourage an accessible app; avoid deep XPath. And the same practical move applies — if an element has no good locator, ask the developers to add an accessibility id. A tiny change to the app to make it testable beats a fragile locator, and it improves accessibility for real users too.

The strictness and waiting still apply

As on the web, a locator should identify one thing, and you wait for elements rather than assuming they are present: await el.waitForDisplayed() before acting on a screen that has just loaded. Everything from the "wait for the condition, never sleep" rule carries over — more so, because devices are slower.

Gestures: the mobile-only actions

The big new category is gestures — touch interactions with no desktop equivalent. You must test these because they are how users actually operate a phone:

• Tap — the basic touch; el.click() taps an element. Long-press, double-tap are variants.

• Swipe / scroll — dragging to move content. Lists are long and the element you want is often off-screen, so you scroll to it before interacting. Modern Appium offers gesture commands:

  // scroll a list until an element is found (UiAutomator2)
  await driver.execute('mobile: scrollGesture', {
    left: 100, top: 300, width: 200, height: 800,
    direction: 'down', percent: 1.0,
  });
  

• Swipe actions — swipe-to-delete, swipe-to-refresh, carousels — common patterns you must exercise.

• Pinch / zoom — two-finger gestures on maps and images.

• Drag and drop, flick.

The key mobile-specific habit: an element off-screen is not absent — you must scroll to bring it into view before acting. A test that fails to find a list item often just needed to scroll first. This is a routine source of confusion for people coming from the web, where the whole page is usually in the DOM.

Beyond the screen: things only mobile has

Native automation also reaches interactions the web-testing module flagged as mobile-specific — and testing them is much of mobile automation's value:

• Permissions dialogs — the OS "Allow location?" prompt; Appium can accept/deny it (often via capabilities like autoGrantPermissions, or by interacting with the system dialog).
• App lifecycle — background and relaunch the app (driver.background(5)), to test the backgrounding/restore behaviour from the permissions-and-interrupts lesson.
• Device conditions — rotate the screen (driver.orientation = 'LANDSCAPE'), toggle network, to test the connectivity and rotation concerns.
• Keyboard — the on-screen keyboard covers fields; you hide it (driver.hideKeyboard()) or account for it.

These are exactly the mobile realities the manual mobile-testing module told you to test — automating them is how you cover device fragmentation, interrupts and connectivity repeatably. This is where mobile automation earns its (considerable) cost: it can drive the phone-specific behaviours a web test never touches.

Check your work

Locate on the native tree with user-facing, stable strategies: accessibility id (~name) is recommended (stable, cross-platform, doubles as an a11y check); id/resource-id (stable, platform-specific); text (readable, breaks on translation); XPath (last resort — slow and brittle on mobile). Prefer accessibility ids; if none exists, have developers add one. Strictness and waiting apply as on the web.

Gestures are the mobile-only actions: tap/long-press/double-tap, swipe/scroll (scroll a list to bring an off-screen element into view before acting — off-screen ≠ absent), swipe-to-delete/refresh, pinch/zoom, drag. Use Appium's mobile: gesture commands.

Beyond the screen: permissions dialogs (autoGrantPermissions/system dialog), lifecycle (background/relaunch), device conditions (rotate, network), keyboard (hideKeyboard). Automating these covers the manual mobile concerns (fragmentation, interrupts, connectivity) repeatably — where mobile automation earns its cost.

Practice

1. Write accessibility-id locators for the AgentPay agent app's email field, sign-in button and a transaction row.
2. Explain why accessibility id is preferred over XPath on mobile, and what you would do if an element lacks one.
3. Write a scroll gesture that brings an off-screen list item into view, then taps it.
4. Explain the "off-screen is not absent" trap and how it differs from web testing.
5. Automate one lifecycle test: background the app for 5 seconds, relaunch, and assert the screen restored correctly.
6. Rotate the device to landscape in a test and assert the layout still works.

Official documentation

• WebdriverIO — Selectors — Locator strategies including accessibility id (~) and why to prefer it.
• WebdriverIO — Action API (gestures) — Taps, swipes and touch gestures in code.
• WebdriverIO — Mobile commands — Lifecycle, orientation, keyboard and device actions.

Next: Android and iOS under one API.