skillZs
★ LIVE SKILL TAGS ★
>>> LIVE SKILLS INDEX <<<
* OPEN SOURCE *
NO LOGIN, NO TRACKING
※ REAL INSTALL DATA ※
← back to all skills
takahirom/android-cli-ui-automation-skill55 installs

android-cli-ui-automation-skill

Automate UI on a connected Android device or emulator via the Android CLI. Use when asked to launch an app, tap a button, enter text, navigate a URL, or drive any on-screen flow to a goal.

How do I install this agent skill?

npx skills add https://github.com/takahirom/android-cli-ui-automation-skill --skill android-cli-ui-automation-skill
view source ↗

Is this agent skill safe to install?

  • Gen Agent Trust Hubpass

    The skill provides powerful Android automation capabilities via the adb and android CLI tools. While these tools are standard for development, they grant the agent significant control over a connected device. The primary security risk is indirect prompt injection, where malicious text within a mobile app or website could trick the agent into performing unintended actions on the device.

  • Socketpass

    No alerts

  • Snykwarn

    Risk: MEDIUM · 1 issue

What does this agent skill do?

Drive the device in a tight launch → observe → act → re-observe loop until the goal is reached. Prefer android layout for structured UI. Use screenshot mode whenever it is faster or more reliable.

If multiple devices are connected, get the serial with adb devices -l. Pass --device=<serial> to android ... and -s <serial> to adb ....

1. Launch

  • URL in a browser: adb -s <serial> shell am start -a android.intent.action.VIEW -d '<url>'
  • Already-installed app: adb -s <serial> shell monkey -p <pkg> -c android.intent.category.LAUNCHER 1
  • APK file: android run --apks=<path> --device=<serial> [--activity=<name>]

2. Observe with android layout

android layout --device=<serial> --pretty --output=/tmp/ui.json

Match targets by text, content-desc, or resource-id. Use the node's center as tap coordinates.

Re-observe after each action with the mode you chose. android layout --device=<serial> --diff is useful when you only need to see what changed.

3. Use screenshot mode when visual targeting is easier

Screenshot mode is often the better choice for:

  • WebView or browser page content
  • Icon-only custom views
  • Canvas, map, or game surfaces
  • Any screen where layout is noisy, incomplete, or slower to interpret than the image
android screen capture --output=/tmp/ui.png --annotate
android screen resolve --screenshot=/tmp/ui.png --string="input tap #N"

Labels are valid only for that screenshot. Capture again after every UI change.

4. Act with adb shell input

adb -s <serial> shell input tap <x> <y>
adb -s <serial> shell input text '<text-with-%s-for-spaces>'
adb -s <serial> shell input swipe <x1> <y1> <x2> <y2> <ms>
adb -s <serial> shell input keyevent <KEYCODE>

Example: adb -s <serial> shell input text 'hello%sworld'

Rules

  • Re-observe after every action. Coordinates and screenshot labels go stale immediately.
  • Clear blockers first: permission dialogs, onboarding sheets, update prompts.
  • Use the observation mode that is clearer: layout for structured UI, screenshot for visual UI.

Notes

  • Don't type URLs into the address bar. Open them with am start -a VIEW -d '<url>'.
  • If the foreground app is unclear, check it with adb -s <serial> shell dumpsys window | rg 'mCurrentFocus|mFocusedApp'.

Add the canonical catalog link to the repository README so users can inspect current installs and available audits. The publishing guide covers the complete discovery path.

<a href="https://skillzs.dev/skills/takahirom/android-cli-ui-automation-skill/android-cli-ui-automation-skill">View android-cli-ui-automation-skill on skillZs</a>