uiautomator2

📖 Read the Chinese version

A simple, easy-to-use, and stable Android automation library.

• QQ Group: 815453846
• Discord: https://discord.gg/PbJhnZJKDd

Users still on version 2.x.x, please check 2to3 before deciding to upgrade to 3.x.x (Upgrade is highly recommended).

How it Works

This framework mainly consists of two parts:

1. Device Side: Runs an HTTP service based on UiAutomator, providing various interfaces for Android automation.
2. Python Client: Communicates with the device side via HTTP protocol, invoking UiAutomator's various functions.

Simply put, it exposes Android automation capabilities to Python through HTTP interfaces. This design makes Python-side code writing simpler and more intuitive.

The Device Side run with a jar file which is also open source. see https://github.com/openatx/android-uiautomator-server-jar

Dependencies

• Android version 4.4+
• Python 3.8+

Installation

pip install uiautomator2

# Check if installation was successful, normally it will output the library version
uiautomator2 version
# or: python -m uiautomator2 version

Install element inspection tool (optional, but highly recommended):

For more detailed usage instructions, refer to: https://github.com/codeskyblue/uiautodev QQ:536481989

pip install uiautodev

# After starting from the command line, it will automatically open the browser
uiautodev
# or: python -m uiautodev

Alternatives: uiautomatorviewer, Appium Inspector

Quick Start

Prepare an Android phone with Developer options enabled, connect it to the computer, and ensure that adb devices shows the connected device.

Open a Python interactive window. Then, input the following commands into the window.

import uiautomator2 as u2

d = u2.connect() # Specify device serial number if multiple devices are connected
print(d.info)
# Expected output
# {'currentPackageName': 'net.oneplus.launcher', 'displayHeight': 1920, 'displayRotation': 0, 'displaySizeDpX': 411, 'displaySizeDpY': 731, 'displayWidth': 1080, 'productName': 'OnePlus5', 'screenOn': True, 'sdkInt': 27, 'naturalOrientation': True}

Example script:

import uiautomator2 as u2

d = u2.connect('Q5S5T19611004599')
d.app_start('tv.danmaku.bili', stop=True) # Start Bilibili
d.wait_activity('.MainActivityV2')
d.sleep(5) # Wait for splash screen ad to disappear
d.xpath('//*[@text="我的"]').click() # Click "My"
# Get fan count
fans_count = d.xpath('//*[@resource-id="tv.danmaku.bili:id/fans_count"]').text
print(f"Fan count: {fans_count}")

Documentation

Connecting to Device

Method 1: Connect using device serial number, e.g., Q5S5T19611004599 (seen from adb devices)

import uiautomator2 as u2

d = u2.connect('Q5S5T19611004599') # alias for u2.connect_usb('123456f')
print(d.info)

To connect on a non-default port (default is 9008):

d = u2.connect('Q5S5T19611004599', port=9009)

Method 2: Serial number can be passed via environment variable ANDROID_SERIAL

# export ANDROID_SERIAL=Q5S5T19611004599
d = u2.connect()

Method 3: Specify device via transport_id

$ adb devices -l
Q5S5T19611004599       device 0-1.2.2 product:ELE-AL00 model:ELE_AL00 device:HWELE transport_id:6

Here you can see transport_id:6.

You can also get all connected transport_ids via adbutils.adb.list(extended=True) Refer to https://github.com/openatx/adbutils

import adbutils # Requires version >=2.9.1
import uiautomator2 as u2
dev = adbutils.device(transport_id=6)
d = u2.connect(dev)

Operating Elements with XPath

What is XPath:

XPath is a query language for locating content in XML or HTML documents. It uses simple syntax rules to establish a path from the root node to the desired element.

Basic Syntax: - / - Select from the root node - // - Select from any position starting from the current node - . - Select the current node - .. - Select the parent of the current node - @ - Select attributes - [] - Predicate expression, used for filtering conditions

You can quickly generate XPath using UIAutoDev.

Common Usage:

d.xpath('//*[@text="私人FM"]').click() # Click element with text "私人FM"

# Syntactic sugar
d.xpath('@personal-fm') # Equivalent to d.xpath('//*[@resource-id="personal-fm"]')

sl = d.xpath("@com.example:id/home_searchedit") # sl is an XPathSelector object
sl.click()
sl.click(timeout=10) # Specify timeout, throws XPathElementNotFoundError if not found
sl.click_exists() # Click if exists, returns whether click was successful
sl.click_exists(timeout=10) # Wait up to 10s

# Wait for the corresponding element to appear, returns XMLElement
# Default wait time is 10s
el = sl.wait()
el = sl.wait(timeout=15) # Wait 15s, returns None if not found

# Wait for element to disappear
sl.wait_gone()
sl.wait_gone(timeout=15)

# Similar to wait, but throws XPathElementNotFoundError if not found
el = sl.get()
el = sl.get(timeout=15)

sl.get_text() # Get component text
sl.set_text("") # Clear input field
sl.set_text("hello world") # Input "hello world" into input field

For more usage, refer to XPath Interface Document

Plugins

• webview: https://github.com/YuYoungG/uiautomator2-webview

To maintain the project's simplicity and extensibility, future plugins will be integrated as third-party libraries.

Operating Elements with UiAutomator API

Element Wait Timeout

Set element search wait time (default 20s)

d.implicitly_wait(10.0) # Can also be modified via d.settings['wait_timeout'] = 10.0
print("wait timeout", d.implicitly_wait()) # get default implicit wait

# Throws UiObjectNotFoundError if "Settings" does not appear in 10s
d(text="Settings").click()

Wait timeout affects the following functions: click, long_click, drag_to, get_text, set_text, clear_text.

Get Device Information

Information obtained via UiAutomator:

d.info
# Output
{'currentPackageName': 'com.android.systemui',
 'displayHeight': 1560,
 'displayRotation': 0,
 'displaySizeDpX': 360,
 'displaySizeDpY': 780,
 'displayWidth': 720,
 'naturalOrientation': True,
 'productName': 'ELE-AL00',
 'screenOn': True,
 'sdkInt': 29}

Get device information (based on adb shell getprop command):

print(d.device_info)
# output
{'arch': 'arm64-v8a',
 'brand': 'google',
 'model': 'sdk_gphone64_arm64',
 'sdk': 34,
 'serial': 'EMULATOR34X1X19X0',
 'version': 14}

Get screen physical size (depends on adb shell wm size):

print(d.window_size())
# device upright output example: (1080, 1920)
# device horizontal output example: (1920, 1080)

Get current App (depends on adb shell):

print(d.app_current())
# Output example 1: {'activity': '.Client', 'package': 'com.netease.example', 'pid': 23710}
# Output example 2: {'activity': '.Client', 'package': 'com.netease.example'}
# Output example 3: {'activity': None, 'package': None}

Wait for Activity (depends on adb shell):

d.wait_activity(".ApiDemos", timeout=10) # default timeout 10.0 seconds
# Output: true or false

Get device serial number:

print(d.serial)
# output example: 74aAEDR428Z9

Get device WLAN IP (depends on adb shell):

print(d.wlan_ip)
# output example: 10.0.0.1 or None

Clipboard

Set or get clipboard content.

• clipboard/set_clipboard

  ```python

  Set clipboard

  d.clipboard = 'hello-world'

  or

  d.set_clipboard('hello-world', 'label')

  Get clipboard

  Depends on input method (com.github.uiautomator/.AdbKeyboard)

  d.set_input_ime() print(d.clipboard) ```

Key Events

• Turn on/off screen

  python d.screen_on() # turn on the screen d.screen_off() # turn off the screen

• Get current screen status

  python d.info.get('screenOn')

• Press hard/soft key

  python d.press("home") # press the home key, with key name d.press("back") # press the back key, with key name d.press(0x07, 0x02) # press keycode 0x07('0') with META ALT(0x02)

• These key names are currently supported:

  • home
  • back
  • left
  • right
  • up
  • down
  • center
  • menu
  • search
  • enter
  • delete ( or del)
  • recent (recent apps)
  • volume_up
  • volume_down
  • volume_mute
  • camera
  • power

You can find all key code definitions at Android KeyEvent

• Unlock screen

  ```python d.unlock()

  This is equivalent to

  1. press("power")

  2. swipe from left-bottom to right-top

  ```

Gesture interaction with the device

• Click on the screen

  python d.click(x, y)

• Double click

  python d.double_click(x, y) d.double_click(x, y, 0.1) # default duration between two clicks is 0.1s

• Long click on the screen

  python d.long_click(x, y) d.long_click(x, y, 0.5) # long click 0.5s (default)

• Swipe

  python d.swipe(sx, sy, ex, ey) d.swipe(sx, sy, ex, ey, 0.5) # swipe for 0.5s (default)

• SwipeExt (Extended functionality)

  ```python d.swipe_ext("right") # Swipe right, 4 options: "left", "right", "up", "down" d.swipe_ext("right", scale=0.9) # Default 0.9, swipe distance is 90% of screen width d.swipe_ext("right", box=(0, 0, 100, 100)) # Swipe within the area (0,0) -> (100, 100)

  In practice, starting swipe from the midpoint for up/down swipes has a higher success rate

  d.swipe_ext("up", scale=0.8)

  Can also use Direction as a parameter

  from uiautomator2 import Direction

  d.swipe_ext(Direction.FORWARD) # Scroll down page, equivalent to d.swipe_ext("up"), but easier to understand d.swipe_ext(Direction.BACKWARD) # Scroll up page d.swipe_ext(Direction.HORIZ_FORWARD) # Scroll page horizontally right d.swipe_ext(Direction.HORIZ_BACKWARD) # Scroll page horizontally left ```

• Drag

  python d.drag(sx, sy, ex, ey) d.drag(sx, sy, ex, ey, 0.5) # drag for 0.5s (default)

• Swipe points

  ```python

  swipe from point(x0, y0) to point(x1, y1) then to point(x2, y2)

  time will be 0.2s between two points

  d.swipe_points([(x0, y0), (x1, y1), (x2, y2)], 0.2) ```

  Often used for pattern unlock, get relative coordinates of each point beforehand (supports percentages). For more detailed usage, refer to this post Using u2 to implement pattern unlock

• Touch and drag (Beta)

  This is a lower-level raw interface, feels incomplete but usable. Note: percentages are not supported here.

  python d.touch.down(10, 10) # Simulate press down time.sleep(.01) # Delay between down and move, control it yourself d.touch.move(15, 15) # Simulate move d.touch.up(10, 10) # Simulate release

Note: click, swipe, drag operations support percentage position values. Example:

d.long_click(0.5, 0.5) means long click center of screen.

Screen Related APIs

• Retrieve/Set device orientation

  The possible orientations:

  • natural or n
  • left or l
  • right or r
  • upsidedown or u (cannot be set)

  ```python

  retrieve orientation. the output could be "natural" or "left" or "right" or "upsidedown"

  orientation = d.orientation

  WARNING: did not pass testing on my TT-M1

  set orientation and freeze rotation.

  notes: setting "upsidedown" requires Android>=4.3.

  d.set_orientation('l') # or "left" d.set_orientation("r") # or "right" d.set_orientation("n") # or "natural" ```

• Freeze/Un-freeze rotation

  ```python

  freeze rotation

  d.freeze_rotation()

  un-freeze rotation

  d.freeze_rotation(False) ```

• Take screenshot

  ```python

  take screenshot and save to a file on the computer, requires Android>=4.2.

  d.screenshot("home.jpg")

  get PIL.Image formatted images. Naturally, you need Pillow installed first

  image = d.screenshot() # default format="pillow" image.save("home.jpg") # or home.png. Currently, only png and jpg are supported

  get OpenCV formatted images. Naturally, you need numpy and cv2 installed first

  import cv2 image = d.screenshot(format='opencv') cv2.imwrite('home.jpg', image)

  get raw jpeg data

  imagebin = d.screenshot(format='raw') with open("some.jpg", "wb") as f: f.write(imagebin) ```

• Dump UI hierarchy

  ```python

  get the UI hierarchy dump content

  xml = d.dump_hierarchy()

  compressed=True: include non-important nodes (default False)

  pretty: format xml (default False)

  max_depth: limit xml depth, default 50

  xml = d.dump_hierarchy(compressed=False, pretty=True, max_depth=30)

  root_in_active=True: dump only the active window root

  xml = d.dump_hierarchy(root_in_active=True) ```

• Open notification or quick settings

  python d.open_notification() d.open_quick_settings()

• Show touch trace on device screen

  ```python

  show touch trace on device screen

  d.show_touch_trace()

  hide touch trace

  d.show_touch_trace(pointer_location=False, show_touches=False) ```

Selector

Selector is a handy mechanism to identify a specific UI object in the current window.

```python

Select the object with text 'Clock' and its className is 'android.widget.TextView'

d(text='Clock', className='android.widget.TextView')