Interact

A decision block that interacts with the on-screen user interface.

If the proceed option is set to Immediately, then the action will be performed on the UI elements currently visible on screen, without pausing the fiber. If set to When UI element appear, then the fiber will pause until the UI element has appeared on screen.

Only use this block as a last resort if there’s no other way of performing the desired automation.

There are two types of interactions:

• Global actions which can be performed at any time and doesn’t require knowledge of the on-screen UI elements, the proceed option is ignored.
• Targeted actions which are performed on an Android UI element, custom graphics used in games will likely not work. The UI element is found by its class, text and/or ID, or if neither is specified, the currently focused UI element (Android 4.1+).

The NO path is executed if the action failed to be performed, or the UI element isn’t visible on screen when using proceed Immediately.

Use the App start or App in foreground block to ensure that the correct activity is shown on screen. A delay between sequential interactions may be needed to ensure the UI has been fully loaded and redrawn.

Note! Interacting with Automate isn’t permitted, for security reasons. The Quick settings action may not work prior to Android 6.

Options

• Proceed — when the fiber should proceed.
• Record interaction — record new UI element interactions.
• Recorded interactions — previously recorded interactions.

Input arguments

• Action — action to perform, default Inspect.
• Package — package name of the app displaying the UI element, default is any.
• UI element class — class of the UI element to interact with, used for targeted actions.
• UI element text — text or description of the UI element to interact with, used for targeted actions, may contain glob pattern.
• UI element ID — ID of UI element to interact with, used for targeted actions. (Android 4.3+)
• Content — generic content, used for the Set text and Focus next HTML element actions.

Output variables

• Inspected content — variable to assign the text content of the inspected element.