Skip to main content

UI Automator

Ultron makes UI Automator actions and assertions much more stable and simple. It wraps both UiObject and UiObject2.

Speed up all UI Automator operations

Ultron operation could be significantly faster then UI Automator one. To accelerate all operations add single line of code in tests precondition.

@BeforeClass
@JvmStatic
fun speedUpAutomator() {
    UltronConfig.UiAutomator.speedUp()
    //or apply the config
    UltronConfig.apply {
        accelerateUiAutomator = true
    }
}

How to use?

Compare following code snippets.

UI Automator

val device = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation())
device.wait(
    Until.findObject(
        By.res("com.atiurin.sampleapp:id", "button1")
    ), 5_000
).click()
val uiObject2 = device.wait(
    Until.findObject(
        By.res("com.atiurin.sampleapp:id", "last_event_status")
    ), 5_000
)
uiObject2.text = "Ultron"
Assert.assertEquals("Ultron", uiObject2.text)

Ultron

byResId(R.id.button1).click()
byResId(R.id.last_event_status).replaceText("Ultron").hasText("Ultron")

The last one looks a little bit better :)

byResId(R.id.button1) actually returns UltronUiObject2.

While the framework tries to execute UI Automator operation, it catches a list of specified exceptions and tries to repeat the operation during the timeout (5 seconds by default). Of course, you are able to customize the list of processed exceptions. It is also possible to specify a custom timeout for any operation. The configuration process for this part of the framework is explained below.

UltronUiObject2 api​

There are factory methods to create UltronUiObject2.

byResId(@IntegerRes resourceId: Int): UltronUiObject2 // specify element with target application resourceId
byText(text: String): UltronUiObject2                 // specify element by its text
by(bySelector: BySelector): UltronUiObject2           // eg by(By.res("com.android.camera2","shutter_button"))

To describe UI element with text or content description use following approach

val textElement = by(By.text("some text"))
val contentDescElement = by(By.desc("Content desc"))

UltronUiObject2 has all methods of standart UiObject2 and also provide a lot of new features.

// data providers
getParent(): UltronUiObject2?                  // return this object's parent, or null if it has no parent
getChildren(): List<UltronUiObject2>           // return a collection of the child elements directly under this object. Empty list if no child exist
getChildCount(): Int
findObject(bySelector: BySelector): UltronUiObject2? // searches all elements under this object and returns the first object to match the criteria
findObjects(bySelector: BySelector): List<UltronUiObject2>  // searches all elements under this object and returns all objects that match the criteria
getApplicationPackage(): String?               // return the package name of the app that this object belongs to
getText(): String?                             // return view.text or null if view has no text
getClassName(): String?                        // return the class name of the view represented by this object
getVisibleBounds(): Rect?                      // return the visible bounds of this object in screen coordinates
getVisibleCenter(): Point?                     // return a point in the center of the visible bounds of this object
getResourceName(): String?                     // return the fully qualified resource name for this object's id
getContentDescription(): String?               // return the content description for this object

//actions
click(duration: Long = 0)                      // A basic click is a touch down and touch up over the same point with no delay.
longClick()
clear()                                        // Clears the text content of an editable field
addText(text: String)                          // Appends text to the text content of an editable field
legacySetText(text: String)                    // Same as replaceText (no longer types key codes since 2.6.6)
replaceText(text: String, verify: Boolean = true) // Sets the text content of an editable field; verify = false skips the result check
drag(dest: Point, speed: Int = DEFAULT_DRAG_SPEED) // Drags object to the specified location
pinchClose(percent: Float, speed: Int = DEFAULT_PINCH_SPEED) // Performs a pinch close gesture on this object
pinchOpen(percent: Float, speed: Int = DEFAULT_PINCH_SPEED)  // Performs a pinch open gesture on this object
// Swipe gestures. [percent] is the length of the swipe as a percentage of the object's size
swipeUp(percent: Float = 0.95F, speed: Int = DEFAULT_SWIPE_SPEED)
swipeDown(percent: Float = 0.95F, speed: Int = DEFAULT_SWIPE_SPEED)
swipeLeft(percent: Float = 0.95F, speed: Int = DEFAULT_SWIPE_SPEED)
swipeRight(percent: Float = 0.95F, speed: Int = DEFAULT_SWIPE_SPEED)
// Scroll gestures. Each one returns Boolean - whether the object can still scroll in that direction
scrollUp(percent: Float = 0.95F, speed: Int = DEFAULT_SCROLL_SPEED): Boolean
scrollDown(percent: Float = 0.95F, speed: Int = DEFAULT_SCROLL_SPEED): Boolean
scrollLeft(percent: Float = 0.95F, speed: Int = DEFAULT_SCROLL_SPEED): Boolean
scrollRight(percent: Float = 0.95F, speed: Int = DEFAULT_SCROLL_SPEED): Boolean
fling(direction: Direction, speed: Int = DEFAULT_FLING_SPEED)         // Performs a fling gesture on this object
perform(actionBlock: UiObject2.() -> Unit, actionDescription: String) // custom action on UiObject2

//asserts
hasText(textMatcher: Matcher<String>)
hasText(text: String)
textContains(textSubstring: String)
textIsNullOrEmpty()
textIsNotNullOrEmpty()
hasContentDescription(contentDescMatcher: Matcher<String>)
hasContentDescription(contentDesc: String)
contentDescriptionContains(contentDescSubstring: String)
contentDescriptionIsNullOrEmpty()
contentDescriptionIsNotNullOrEmpty()
isCheckable()
isNotCheckable()
isChecked()
isNotChecked()
isClickable()
isNotClickable()
isEnabled()
isNotEnabled()
isFocusable()
isNotFocusable()
isFocused()
isNotFocused()
isLongClickable()
isNotLongClickable()
isScrollable()
isNotScrollable()
isSelected()
isNotSelected()
isDisplayed()
isNotDisplayed()
assertThat(assertBlock: UiObject2.() -> Boolean, assertionDescription: String) // custom assertion of UiObject2

//------ general ------ 
withTimeout(timeoutMs: Long)                     // set custom timeout for operations
withResultHandler(resultHandlerBlock)            // set custom result handler and process operation result 
withAssertion(assertion: OperationAssertion)     // define custom assertion of action success
withAssertion(name: String = "", isListened: Boolean = false, block: () -> Unit)
withName(name: String)                           // custom element name shown in logs, exceptions and Allure steps
withMetaInfo(meta: Any)                          // associate any custom info with the element
isSuccess(action: UltronUiObject2.() -> Unit): Boolean // get the result of any operation as Boolean

UltronUiObject api​

As it was mentioned before Ultron wraps UiObject too. There is a set of static methods to create UltronUiObject.

uiResId(@IntegerRes resourceId: Int): UltronUiObject // specify element with target application resourceId
ui(uiSelector: UiSelector): UltronUiObject

It has all methods of standart UiObject and also provide a lot of new features. As UltronUiObject has almost the same api as UltronUiObject2 we don't list it.

Text actions are verified​

addText, replaceText, clear and legacySetText of UltronUiObject2, and legacyAddText of UltronUiObject, succeed only when the object's text becomes the expected value. UI Automator's own setText does not report a refused set-text action, so previously a text action on a non-editable object (a button, a layout that wraps the real field) passed without changing anything. Now such an action is retried until the operation timeout and then fails with the expected and the actual text and the object's class:

Text of android.widget.LinearLayout was expected to become 'machine' but is ''

The check takes into account how UI Automator reports text fields:

  • An empty field reports its hint as its text. addText treats it as empty, and clear accepts it. A field whose own text is exactly equal to its hint is treated as empty too.
  • A password field (the node's isPassword flag, set by Android for a PasswordTransformationMethod) reports its displayed, masked text. Its text is compared by length only, whatever the mask character is. addText refuses to append to a non-empty password field because its current text cannot be read back; use replaceText. A field masked by a custom transformation that is not a PasswordTransformationMethod is not flagged as a password by Android, so its text is compared exactly and the action fails with the masked value in the message.
  • addText reads the current text once and sets current + text on every retry, so a retry never appends twice.

legacySetText (UltronUiObject2) and legacyAddText (UltronUiObject) used to type key codes through the hidden legacySetText of UI Automator, which UI Automator 2.3.0 removed. Since 2.6.6 they use the set-text action: legacySetText behaves as replaceText, and legacyAddText appends as addText does. Ultron 2.6.6 depends on UI Automator 2.3.0.

For a custom field whose reported text never equals the text that was set (a custom mask, formatting), call replaceText(text, verify = false): the check is skipped and the action succeeds as soon as the text is set, even if the object ignored it.

Best practice​

Specify page elements as properties of PageObject class.

object SomePage : Page<SomePage>() {
    private val button = byResId(R.id.button1)
    private val eventStatus = byResId(R.id.last_event_status)
}

Use this properties in page steps

object SomePage : Page<SomePage>() {
    //page elements
    fun someUserStepOnPage(expectedEventText: String){
         button.click()
         eventStatus.hasText(expectedEventText)
    }
}

Custom timeout for any operation​

byResId(R.id.last_event_status).withTimeout(10_000).isDisplayed()

There are 2 ways of using custom timeout:

  • Specify it for page property and it will be applied for all operations with this element
object SomePage : Page<SomePage>() {
    private val eventStatus = byResId(R.id.last_event_status).withTimeout(10_000)
}
  • Specify it inside special step there the element operation could take more time. This timeout value will be applied only once for single operation.
object SomePage : Page<SomePage>() {
    fun someLongUserStep(expectedEventText: String){
         longRequestButton.click()
         eventStatus.withTimeout(20_000).hasText(expectedEventText)
    }
}

Boolean operation result​

There is isSuccess method that allows us to get the result of any operation as boolean value. In case of false it could be executed to long (5 sec by default). So it's resonable to specify custom timeout for some operations.

val isButtonDisplayed = byResId(R.id.button).isSuccess { withTimeout(2_000).isDisplayed() }
if (isButtonDisplayed) {
    //do some reasonable actions
}

Extend framework with your own action and assertion​

It's described in another page here