SuperAPI AI Directory
Partner中文AI newsFavorites
Skillsdriver-ui-tests

driver-ui-tests

Testing & quality

View repository

Updated 10-16 07:20

About this skill

Driver UI Tests Guide

Guidelines for writing UI tests using IDE Starter and UI Driver frameworks.

Common Imports

// Driver core
import com.intellij.driver.client.Driver
import com.intellij.driver.sdk.waitForProjectOpen
import com.intellij.driver.sdk.advancedSettings

// Test utilities
import com.intellij.driver.tests.utils.waitForIndicators
import com.intellij.driver.tests.utils.Plugin
import com.intellij.driver.tests.utils.PluginInstaller
import com.intellij.driver.tests.utils.Plugins

// IDE Starter framework import com.intellij.ide.starter.driver.runIdeTest import com.intellij.ide.starter.ide.IDETestContext import com.intellij.ide.starter.models.IDEStartResult import com.intellij.ide.starter.models.VMOptions import com.intellij.ide.starter.runner.IDERunContext import com.intellij.ide.starter.runner.Starter import com.intellij.ide.starter.utils.catchAll

// Extended test infrastructure import com.intellij.ide.starter.extended.allure.AllureHelperExtended.step import com.intellij.ide.starter.extended.allure.Subsystems import com.intellij.ide.starter.extended.engine.newTestContainerExtended import com.intellij.ide.starter.extended.engine.TestContainerExtended import com.intellij.ide.starter.extended.license.StagingLicenseGenerator.licenseProductCode import com.intellij.ide.starter.extended.loadMetadataFromServer import com.intellij.ide.starter.extended.setupTestMetadataSchemeWithGroupsFromCode

// Test framework import com.intellij.testFramework.TestApplicationManager


## Test Structure
- Tests use JUnit 5 with an IDE Starter framework and UI Driver framework (`community/platform/remote-driver`)
- Test case projects are represented by the `com.intellij.ide.starter.models.TestCase` see `src/com/intellij/ide/starter/project`
- Tests run against specific IDE (`community/tools/intellij.tools.ide.starter/src/com/intellij/ide/starter/ide/IdeProductProvider.kt`)

## Test project examples

See `tests/intellij.ide.starter.extended/src/com/intellij/ide/starter/extended/data/cases`

## Page Object Pattern or UiComponent

Page objects extend `UiComponent` with `ComponentData` constructor:

```kotlin
class MyPageObject(data: ComponentData) : UiComponent(data) {
    val myButton = x { byAccessibleName("Button Name") }
    val myPanel = x { byClass("PanelClassName") }

    fun clickMyButton() {
        myButton.click()
    }
}

// Extension function on Finder to create the page object
fun Finder.myPageObject(): MyPageObject = x(
    xQuery { byAccessibleName("Root Element Name") }, MyPageObject::class.java
)

// Use specific ui component to specify the context of the search
fun AnotherPageObject.myPageObject(): MyPageObject = x(
  xQuery { byAccessibleName("Root Element Name") }, MyPageObject::class.java
)

Element Selectors

Selector Usage
byAccessibleName("name") Find by accessible name attribute
byClass("ClassName") Find by Swing/AWT class name
byVisibleText("text") Find by visible text content

UI test examples

See directory tests/remote-driver-tests

Common UI Components

See directory community/platform/remote-driver/test-sdk/src/com/intellij/driver/sdk/ui

Scoping Element Searches

When multiple elements match a selector, scope to a parent element:

// BAD - will fail if multiple InstallButtons exist
ui.x { byClass("InstallButton") }.click()

// GOOD - scope to a parent element first
val detailPane = ui.x { byClass("PluginDetailsPageComponent") }
detailPane.x { byClass("InstallButton") }.click()

Finding toolbar / title-bar actions

Toolbar and tool-window title actions that show their text (presentation.putClientProperty(ActionUtil.SHOW_TEXT_IN_TOOLBAR, true)) render as ActionButtonWithText, not ActionButton. The SDK actionButton(text) helper searches @class='ActionButton' only, so it silently never matches them. Match by visible text across both variants:

// Matches both icon-only and text-bearing action buttons
fun Finder.statusButton(text: String) =
  x("//div[(@class='ActionButtonWithText' or @class='ActionButton') and @visible_text='$text']")

The visible text is itself a reliable assertion signal — you usually do not need to read the backing service/state.

Keyboard Interactions

element.keyboard { typeText("search text") }
ui.keyboard { key(KeyEvent.VK_ENTER) }
ui.keyboard { hotKey(KeyEvent.VK_META, KeyEvent.VK_COMMA) }  // Cmd+,

Writing Tests

Required Annotations

Every UI test must have the following annotations at the class level:

Annotation Purpose TestOps Custom Field
@Subsystems.* Categorizes the test by subsystem Subsystem
@Features.* Specifies the feature being tested Feature
@Components.* Identifies the component under test Component

For tests linked to TestOps test cases, also add:

  • @AllureId("test_case_id") - Links the test to the TestOps test case ID

The annotation values should match the corresponding TestOps custom fields (Subsystem, Feature, Component).

Available annotations: See tests/intellij.ide.starter.extended.allure/src/com/intellij/ide/starter/extended/allure/Annotations.kt

Example with TestOps test case:

@Subsystems.Java
@Features.Completion
@Components.Editor
class MyTestFromTestOps {

    @Test
    @AllureId("318541")  // Required when test case exists in TestOps
    fun `my test from testops`(testInfo: TestInfo) {
        // ...
    }
}

Example for new test (not yet in TestOps):

@Subsystems.UI
@Features.PluginManager
@Components.Miscellaneous
class MyNewTest {

    @Test
    fun `my new test`(testInfo: TestInfo) {
        // ...
    }
}

Basic Test Structure

@Subsystems.Java
@Features.Completion
@Components.Editor
class MyTest {
    val testCase = TestCase(IdeProductProvider.IU, myProject)

    @Test
    @AllureId("123456")  // Required if test case exists in TestOps
    fun `my test name`(testInfo: TestInfo) {
        val context = Starter.newContext(testName = "TestName", testCase = testCase)

        context.applyVMOptionsPatch {
            addSystemProperty("ide.ui.non.modal.settings.window", "true")
        }

        context.runIdeTest(testName = testInfo.displayName) {
            waitForIndicators(5.minutes)  // Wait for indexing to complete

            step("Step description") {
                // Test actions here
            }
        }
    }
}

Waiting for Project Import and Indexing

Always wait for indicators at the start of your test:

waitForIndicators()

This ensures the project is fully imported and indexed before interacting with the IDE.

Opening Files

Use openFile instead of UI-based file navigation:

// GOOD - Direct and reliable
openFile(relativePath = "src/Main.java")

// AVOID - UI-based approach is slower and more fragile
invokeAction("GotoFile", now = false)
ui.keyboard { typeText("Main.java") }
ui.keyboard { key(KeyEvent.VK_ENTER) }

invokeAction: now Parameter

The now parameter controls whether the action completes before continuing:

// now = true: Waits for action to complete (use when keyboard input follows)
invokeAction("ToggleBookmarkWithMnemonic", now = true)
ui.keyboard { key(KeyEvent.VK_1) }  // This input goes to the bookmark dialog

// now = false: Returns immediately (use when waiting for UI to appear)
invokeAction("ShowSettings", now = false)
ui.x { byClass("SettingsDialog") }.shouldBe { present() }

Rule: Use now = true when the next step is keyboard input to prevent input going to the wrong component. Rule: Use now = false when you expect to the UI dialog to appear.

Custom Wait Conditions

Use waitFor to wait for specific conditions:

waitFor("description of what we're waiting for", 30.seconds) {
    ui.x { byClass("MyComponent") }.present()
}

waitFor("text to appear", 10.seconds) {
    ui.x { byClass("Tree") }.hasText("expected text")
}

Reading IDE state via @Remote

To read state from a service or model in the IDE under test, declare a @Remote interface and call it via driver.service(...) / driver.utility(...).

  • Plugin classes need the plugin field. Without it the class resolves against the platform/core classloader → DriverIllegalStateException: No such class '<fqn>' in plugin null.
    • Class in a plugin content module: @Remote("<fqn>", plugin = "<plugin.id>/<content.module>") (e.g. com.intellij.figma/intellij.figma.core).
    • Class in the main / embedded plugin module: @Remote("<fqn>", plugin = "<plugin.id>").
  • Method dispatch resolves against the DECLARED @Remote class, not the runtime object. A method declared on a sealed/abstract supertype ref is "not found" at runtime — declare it on the concrete subtype, or expose it via a top-level type. (The jvm-class-name injection also cannot resolve a nested Foo$Bar name → a cosmetic "Cannot resolve class" inspection error; prefer top-level types.)
  • Add the plugin module as a TEST dependency so the FQNs resolve for code-insight.
@Remote("com.example.MyAppService", plugin = "com.example.myplugin/com.example.myplugin.core")
interface MyAppServiceRef {
  fun getConfig(): MyConfigRef
}
// driver.service(MyAppServiceRef::class).getConfig()...

Enabling a registry flag at startup

Seed a registry key before the IDE starts with a -D VM option — RegistryValue falls back to System.getProperty. Required when a startup ProjectActivity or ToolWindowFactory.shouldBeAvailable reads the flag (setting it via the driver after start is too late):

context.applyVMOptionsPatch {
  addSystemProperty("my.feature.enabled", "true")
}

Driving a real browser (Playwright)

Playwright runs in the test JVM, alongside the driver-driven IDE (both on localhost) — useful when the IDE's client is a web app/plugin. page.onConsoleMessage { ... } captures the page and its iframes (a strong diagnostic). Put custom screenshots/files under context.paths.testHome.resolve("log") so they are collected as test artifacts. See plugins/figma/integrationTests for a full example.

Running Tests from Terminal

Driver tests require a fully built IDE. There are several ways to run them:

Option 1: Using tests.cmd (Recommended)

The tests.cmd script builds the IDE from sources and runs tests. Recommended for dev server mode.

Example:

./tests.cmd \
  --module intellij.driver.tests \
  --test com.intellij.driver.tests.idea.java.FindAndGoToTest

Key parameters:

  • --test - fully qualified test class name (or pattern)
  • --module intellij.driver.tests - required for driver tests

Example with specific test:

./tests.cmd \
  --module intellij.driver.tests \
  --test com.intellij.driver.tests.idea.ultimate.httpclient.BuiltInHttpClientBrotliCompressionUiTest

Debugging Test Failures

Output Locations

After test failure, check:

  • UI hierarchy: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/log/ui-hierarchy/ui.html
  • IDE log: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/log/idea.log
  • Screenshots: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/log/screenshots/
  • Exceptions: out/ide-tests/tests/{IDE-version}/{TestName}/{test-method}/error/

Inspect the LIVE UI hierarchy, not just the post-mortem file

The ui-hierarchy/ui.html and full-screen.png written on failure are captured after useDriverAndCloseIde tears the IDE down — by then the session has ended and panels often revert to an empty/welcome state, so they can be misleading. Two better sources:

  • Heartbeat screenshot log/screenshots/001_heartbeat/ — captured mid-run, shows the real state during the wait.
  • Live UI hierarchy server — while the IDE is up, the component tree is browsable at http://localhost:<port>/api/remote-driver/ (the harness sets -Dexpose.ui.hierarchy.url=true; the port is logged at startup as UI Hierarchy: http://localhost:<port>/api/remote-driver/). To inspect interactively, park the test at the point of interest — temporarily raise a waitFor timeout (e.g. to 20.minutes) — and curl/open that URL while the IDE stays alive. Each node exposes class (simple), javaclass (FQN, incl. Outer$Inner for inner classes), visible_text, and accessiblename; read these to build a reliable matcher instead of guessing from source.

Common Issues

  1. Element Not Found: Check UI hierarchy HTML for the correct accessible name or class
  2. Multiple Elements Match: Scope search to parent element

Critical Rules

  • Never use Thread.sleep() or delay() - Driver framework automatically waits for UI elements
  • Wrap test logic in step("description") { } for better logs
  • Verify assertions actually fail – Comment out the action being tested and confirm the test fails. If it still passes, your assertion is too weak.
  • Use common UI components – create new if necessary
  • Always check UI hierarchy to understand the UI state when a test fails