Skip to main content

Vendor evaluation: mobile-next suite vs Maestro

Due-diligence report, 2026-07-22. Source-level review of all mobile-next org repos + independent signals.

Executive summary​

  • The famous repo is not the useful one. mobile-mcp (5.5k★) is a stateless wrapper — every MCP call is a fresh adb/mobilecli shell-out (and on real iOS a create→act→delete WebDriverAgent session per call). Its Android hierarchy is stock adb shell uiautomator dump XML filtered to a flat list. Adopting it wholesale would not fix Rover's latency and would regress tree quality. Rule it out as the integration surface.
  • The actually-valuable asset is mobilecli — a Go binary with a real persistent daemon (mobilecli server --daemon: JSON-RPC 2.0 over HTTP /rpc + WebSocket /ws + MJPEG/AVC /stream, ~40 methods, shipped OpenRPC spec) plus devicekit-android, an install-free on-device UiAutomation JSON hierarchy dumper with H.264/MJPEG screen streaming. This stack can deliver a real ~10× latency win and cleaner multi-device scaling.
  • But two things blunt the case. (1) The same latency win is largely reachable without mobile-next by moving Rover off per-maestro test CLI onto Maestro's own persistent session (Maestro MCP / continuous driver — Maestro is already an on-device gRPC-server design). (2) Licensing is the decisive blocker: mobilecli, devicekit-android, and devicekit-ios are all FSL-1.1 (Functional Source License) with an explicit "Competing Use" carve-out, and Aucert is exactly the "substantially similar functionality" competitor it excludes. Maestro is clean Apache-2.0.
  • Recommendation: fix the incumbent first (persistent Maestro session), run a 1-day head-to-head, and — since the founders reached out — open a commercial-licensing/partnership conversation in parallel. Adopt mobilecli selectively (capture path only, behind Rover's driver interface) only if it beats Maestro-persistent on the benchmark and we secure a commercial license. Keep iOS on the current path.

1. The suite map​

22 repos in the org; the automation suite is 5 libraries + a hosted cloud:

mobilewright (TS, Apache-2.0)     mobile-mcp (TS, Apache-2.0)
 Playwright-style test framework    MCP server for LLM agents
        │  persistent WS JSON-RPC          │  Android: bypasses mobilecli → raw adb
        │                                   │  iOS sim/remote: shells mobilecli per call
        └───────────────┬───────────────────┘
                        ▼
             mobilecli (Go, FSL-1.1)   ← the real universal driver
             • CLI + `server --daemon` (JSON-RPC/WS/stream, OpenRPC-specced)
             • Android: adb input/screencap + devicekit-android for dump/stream
             • iOS: danielpaulus/go-ios (upstream) + devicekit-ios (WDA) for UI
        ┌───────────────┴───────────────┐
        ▼                               ▼
 devicekit-android (Kotlin, FSL-1.1)  devicekit-ios (Swift, FSL-1.1)
  on-device APK: UiAutomation JSON     WDA-internals repackaged behind Swift
  dump + MJPEG/AVC stream              JSON-RPC; XCUITest bundle; code-signing
                        │
        Mobile Next Cloud / "Mobile Fleet" (hosted pay-per-minute device farm)
        — the commercial product the FSL protects.
RepoLang / LicenseStarsRole
mobilecliGo / FSL-1.1 (npm wrapper mislabels MIT)223Universal driver + persistent daemon; ~40-method JSON-RPC (server/dispatch.go), OpenRPC spec (docs/openrpc.json)
devicekit-androidKotlin / FSL-1.119On-device APK: UiAutomation JSON hierarchy (UiTreeSerializer.kt), MJPEG/H.264 streaming; dump-only resident socket; no tap/input by design
devicekit-iosSwift / FSL-1.116WDA private-API internals behind Swift JSON-RPC; XCUITest bundle :12004; needs code-signing
mobile-mcpTS / Apache-2.05,533MCP server; stateless per-call dispatch; Android = raw adb
mobilewrightTS / Apache-2.0300Playwright-style framework; persistent WS JSON-RPC to mobilecli
go-ios (fork)Go / MIT1Dormant fork, unused (mobilecli imports upstream)

Company: Mobile Next HQ, Inc. (mobilenext.ai), monetizing a hosted device cloud ("Mobile Fleet").

2. Deep comparison vs. Maestro​

DimensionMaestro (incumbent)mobile-next (mobilecli daemon)Verdict for Rover
Session modelOn-device Netty gRPC server (hardcoded :7001); Rover's pain is the per-action CLI spawn, not the driver design; persistent modes exist (Maestro MCP, continuous driver)Long-lived host daemon; but each hierarchy dump still runs adb shell am instrument fresh (~1–2 s, 10× retry on null root)Both can be persistent — the fix is "stop spawning a CLI per action," achievable on either
Per-action latencyRover-today: 8–15 s/tap, ~5 s/dump, 30–60 s/settled step. Maestro-persistent: sub-second tapsTap ~0.3–0.8 s, dump ~1–2 s, screenshot ~free via /stream; settled step ~2–5 s~10× over Rover-today, parity with Maestro-persistent
Hierarchy qualityIn-process UiAutomator / a11y bridge; nested, resource-ids, boundsdevicekit ViewTreeDump: UiAutomation over all windows, incl. not-important views; note: mobilecli's normalization drops clickable/enabled/scrollable — consume raw devicekit JSONParity (same a11y source); mobile-mcp's flat list = regression
WebViewa11y virtual nodesSame; JVMTI DOM/JS eval only on debuggable appsParity on real apps
iOSXCUITest/WDAgo-ios + devicekit-ios (younger; protocol inconsistency mid-migration; signing pain)No advantage today
Concurrency robustnessHardcoded :7001 wedges under concurrent clients (issues #3065/#3239/#1570) — our scarDynamic per-device forwards, stateless input → parallel-friendly; but stale-connection stalls (#358, open), no client-side timeoutsDifferent failure modes, not fewer; must add own watchdog
Device farmPort collisions across sessions--device everywhere, dynamic forwards, hosted FleetEdge mobile-next for parallel scale
Split APKManualNo install-multiple eitherParity — our own handling carries over
Screenshotscreencap per callPer-call or persistent MJPEG/AVC streamEdge mobile-next
MaturityLarge project, broad adoption2 devs ≈ 80% of commits; 1-2 maintainers on devicekits; minimal independent proof; security history (#301 path-traversal RCE fixed; #330 default-on telemetry)Edge Maestro
LicenseApache-2.0FSL-1.1 non-compete on every load-bearing pieceDecisive edge Maestro

3. What we'd gain (mobilecli, not mobile-mcp)​

  1. Kill host-side per-action process spawn (the 30–60 s step lever).
  2. Near-free screenshots via persistent stream.
  3. Cleaner multi-device parallelism (device-farm substrate).
  4. One machine-readable JSON-RPC contract (OpenRPC → generated Kotlin client).
  5. All-windows install-free UiAutomation JSON capture (dialogs, IME, popups).
  6. WebView DOM/JS on our own debug builds.
  7. Engaged founders, daily release cadence.

4. NOT worth switching for​

  1. mobile-mcp itself — stateless per-call + flat tree = same latency class as our pain, worse capture.
  2. "Better hierarchy" — same a11y source as Maestro, no leap.
  3. WebView on real apps (debuggable-only DOM).
  4. Split-APK (parity).
  5. iOS (younger, mid-migration, signing pain).
  6. Pure latency — largely fixable via persistent Maestro with zero license risk.
  7. Robustness — trades wedge for stalls; not proven better.

5. Worth considering selectively​

  • Capture path only: devicekit-android's resident dump socket + MJPEG stream as Rover's fast hierarchy/screenshot source, behind our driver interface; Maestro keeps input/flows. License-gated.
  • mobilecli daemon as full sidecar during a transition. License-gated.
  • devicekit-ios revisit post-1.0.
  • Hosted Mobile Fleet as a real-device breadth stopgap (SaaS dependency; the exact product the FSL protects).

6. Integration sketch + risks (mobilecli sidecar)​

Sidecar daemon per worker/host (fits Tilt/K8s, ADR-024); Kotlin client generated from OpenRPC; MobilecliDriver alongside MaestroDriver behind the existing driver interface, feature-flagged; consume raw devicekit JSON for the tree; /stream for frames.

Risks in priority order: (1) FSL-1.1 Competing Use — "same or substantially similar functionality… in a commercial product" is us; legal review + commercial license required before any production embed (npm MIT mislabel adds ambiguity, not comfort). (2) Reliability gaps (#358 stalls; wrap every call in timeout + health-check + re-attach watchdog; per-dump am instrument ~1–2 s with transient null-root). (3) iOS immaturity/protocol inconsistency. (4) Bus factor — pin, vendor, be ready to fork (FSL permits internal modification). (5) Telemetry — audit + disable phone-home. (6) Security history — sandbox the daemon. (7) Split-APK stays ours.

7. One-day benchmark plan (beans)​

Rigs: A Maestro-persistent (MCP/continuous driver) · B mobilecli daemon · C current per-CLI driver (baseline). Metrics: taps/min over a 100-step walk; hierarchy p50/p95 over 100 dumps; screenshot per-call vs stream; settled-step wall time; tree-quality diff on the same 10 screens through our normalize pipeline (node count, resource-id + interactable coverage, identity-signature stability); 3-way concurrency for 30 min (wedge/stall rate); induced-stall recovery time. Go/no-go for B: settled step < 5 s; coverage within a few % of Maestro; identity signatures stable; survives concurrency with automatic recovery. Tree destabilization = no-go regardless of speed.

8. Bottom line​

  1. Don't switch to mobile-mcp (regression).
  2. Fix the incumbent first — persistent Maestro session, re-measure; likely recovers most of the ~10× with zero license risk.
  3. Engage the founders on two tracks: the 1-day head-to-head + a commercial-licensing/partnership conversation (the FSL means friendly OSS adoption isn't actually available for the load-bearing pieces).
  4. Adopt mobilecli selectively only if it clearly beats Maestro-persistent on the benchmark and a commercial license is secured; keep fork rights.
  5. iOS stays on the current path.

Net: the tech is real and well-architected — the persistent JSON-RPC daemon + all-windows UiAutomation capture is exactly the right shape for Rover — but it delivers parity with Maestro-persistent, not a categorical leap, and carries a non-compete license on every load-bearing component. "Fix Maestro + negotiate if the benchmark justifies it" is the rational path. Any FSL determination goes to counsel before production use.