Scout UI Testing
Testing & Qualityscout-ui-testingScout UI Testing is an Agent Skill for testing & quality published in the elastic/kibana repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Use when creating, updating, debugging, or reviewing Scout UI tests in Kibana (Playwright + Scout fixtures), including page objects, browser authentication, parallel UI tests (spaceTest/scoutSpace), a11y checks, and flake control.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/elastic/kibana/blob/HEAD/.agents/skills/scout-ui-testing/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/scout-ui-testing, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Scout UI Testing
Pick the right test mode
- Sequential UI:
<module-root>/test/scout*/ui/tests/**/*.spec.ts. - Parallel UI:
<module-root>/test/scout*/ui/parallel_tests/**/*.spec.tsand (recommended) usespaceTest+scoutSpace(one Kibana space per worker). If you run withworkers > 1but keep usingtest, you won't get space isolation. - Use the Scout package that matches the module root:
src/platform/**orx-pack/platform/**->@kbn/scoutx-pack/solutions/observability/**->@kbn/scout-obltx-pack/solutions/search/**->@kbn/scout-searchx-pack/solutions/security/**->@kbn/scout-security
Imports
- Test framework + tags:
import { tags } from '@kbn/scout';(or the module's Scout package) - Test fixture:
import { test } from '../fixtures';(orimport { test } from '@kbn/scout';when not extending) - Assertions:
import { expect } from '@kbn/scout/ui';(or@kbn/scout-oblt/ui, etc.) — not from the main entry expectis not exported from the main@kbn/scoutentry. Use the/uisubpath for UI tests.
Non-negotiable conventions
- Tags are required: Scout validates UI test tags at runtime. Ensure each test has at least one supported tag (typically by tagging the top-level
test.describe(...)/spaceTest.describe(...), e.g.tags.deploymentAgnostic,tags.stateful.classic, ortags.performance). - No
@in test titles: Playwright treats@wordin test/describe titles as tags. Do not use@followed by word characters in titles (e.g.,@timestamp,@elastic). This causes Scout tag validation to fail with "Unsupported tag(s) found". Rephrase the title instead (e.g., usetimestamp fieldinstead of@timestamp). - Prefer one suite per file: keep a single top-level
test.describe(...)(sequential) orspaceTest.describe(...)(parallel) and avoid nesteddescribeblocks where possible. - UI actions live in page objects; assertions stay in the spec.
- Use APIs for setup/teardown: prefer
apiServices/kbnClient/esArchiverin hooks over clicking through the UI.
Auth (UI)
- Use
browserAuth— available methods:loginAsAdmin(),loginAsPrivilegedUser(),loginAsViewer(),loginAs(role),loginWithCustomRole(role). - Prefer least privilege: use
loginAsViewer()orloginWithCustomRole()overloginAsAdmin(). - Avoid
loginAsAdmin()unless the test is explicitly about admin-only behavior.
Page objects (UI)
- Prefer
page.testSubj.locator(...), role/label locators; avoid brittle CSS. - Keep selectors + interactions inside the page object class. Do not use
expectassertions in page objects — usewaitForSelectorfor waiting on elements. Assertions belong in test specs only. - Keep route mocks out of page objects — page objects are for UI interactions only. Put
page.route()mocks in a dedicatedfixtures/mocks.tsfile as standalone functions that acceptpageas a parameter. Seecloud_security_posture/test/scout_cspm_agentless/ui/fixtures/mocks.tsfor the reference pattern. - Don't make API calls from page objects (use
apiServices/kbnClientin hooks instead). - Register plugin page objects by extending the
pageObjectsfixture intest/scout*/ui/fixtures/index.ts. - Use
readonlyclass fields for static locators — assign them in the constructor, not as getter methods. Use methods only for parameterized locators/actions. SeeDashboardAppinkbn-scoutfor the reference pattern. - Scout provides EUI component wrappers for stable interactions with common EUI widgets:
EuiComboBoxWrapper,EuiDataGridWrapper,EuiSelectableWrapper,EuiCheckBoxWrapper,EuiFieldTextWrapper,EuiCodeBlockWrapper,EuiSuperSelectWrapper,EuiToastWrapper. Import them from@kbn/scoutand use them as class members in page objects. - Avoid
.first(),.nth(),.last()— theplaywright/no-nth-methodslint rule flags these. Instead, usedata-test-subjattributes or other targeted selectors. If the component lacks adata-test-subj, add one rather than disabling the rule. - Do not disable eslint rules — avoid
eslint-disablecomments in test files. Fix the underlying issue (e.g., use targeted selectors instead of positional ones, adddata-test-subjto the components) rather than suppressing the lint rule.
Parallel UI specifics (spaceTest)
- Use
spaceTestso you can accessscoutSpacefor worker-isolated saved objects + UI settings. - Pre-ingest shared ES data in
parallel_tests/global.setup.tsviaglobalSetupHook(...).- Only worker fixtures are available there (no
page,browserAuth,pageObjects).
- Only worker fixtures are available there (no
- Reset Elasticsearch/Kibana state once after the suite via
globalTeardownHook(...)inparallel_tests/global.teardown.ts(optional, opt-in by file presence). For state that does need resetting, useesClient/kbnClient/apiServices. Seereferences/scout-ui-parallelism.md. - Cleanup space-scoped mutations in
afterAll(scoutSpace.savedObjects.cleanStandardList(), unset UI settings you set).
Extending fixtures
Most modules extend the base test (or spaceTest) in test/scout*/ui/fixtures/index.ts to add custom page objects and auth helpers:
import { test as baseTest } from '@kbn/scout'; // or the module's Scout package
import type { ScoutTestFixtures, ScoutWorkerFixtures, ScoutPage } from '@kbn/scout';
class MyPluginPage {
constructor(private readonly page: ScoutPage) {}
async goto() { await this.page.gotoApp('myPlugin'); }
}
interface ExtendedFixtures extends ScoutTestFixtures {
pageObjects: ScoutTestFixtures['pageObjects'] & { myPlugin: MyPluginPage };
}
export const test = baseTest.extend<ExtendedFixtures, ScoutWorkerFixtures>({
pageObjects: async ({ pageObjects, page }, use) => {
await use({ ...pageObjects, myPlugin: new MyPluginPage(page) });
},
});
Tests then import from local fixtures: import { test } from '../fixtures';
Multi-step flows with test.step()
Use test.step(...) to group related actions within a single test. Steps appear in Playwright's trace viewer and HTML report, making failures easier to debug without splitting into many small tests:
test('creates and verifies a dashboard', async ({ pageObjects, page }) => {
await test.step('create dashboard', async () => {
await pageObjects.dashboard.create('My Dashboard');
});
await test.step('verify dashboard appears in list', async () => {
await expect(page.testSubj.locator('dashboardTitle')).toHaveText('My Dashboard');
});
});
Waiting + flake control
- Don’t use
page.waitForTimeout. Wait on a page-ready signal (loading indicator hidden, container visible,expect.pollon element counts). - When an explicit wait is needed, prefer
locator.waitFor({ state: 'visible' })over a barelocator.waitFor(). The two are equivalent (visibleis the default state), but stating it keeps the intent explicit and consistent with RTL-style readiness checks. - If selectors aren’t stable, add
data-test-subj(Scout uses it as thetestIdAttribute). - Some locators are restricted by
@kbn/eslint/scout_no_locators(e.g.globalLoadingIndicator). Don’t use them in tests or page objects for app loading state management; rely on Playwright auto-waiting and page-ready signals instead.
A11y checks (optional, high value)
- Use
page.checkA11y()at a few stable checkpoints (landing pages, modals/flyouts). - Prefer
includescoped checks; assertviolationsis empty.
Run / debug quickly
- Use either
--configor--testFiles(they are mutually exclusive). - Run by config:
node scripts/scout.js run-tests --arch stateful --domain classic --config <module-root>/test/scout*/ui/playwright.config.ts(or.../ui/parallel.playwright.config.tsfor parallel UI) - Run by file/dir (Scout derives the right
playwright.config.tsvsparallel.playwright.config.ts):node scripts/scout.js run-tests --arch stateful --domain classic --testFiles <module-root>/test/scout*/ui/tests/my.spec.ts - For faster iteration, start servers once in another terminal:
node scripts/scout.js start-server --arch stateful --domain classic [--serverConfigSet <configSet>], then run Playwright directly:node scripts/playwright test --config <...> --project local --grep <tag> --headed. run-testsauto-detects custom config sets from.../test/scout_<name>/...paths.start-serverhas no Playwright config to inspect, so pass--serverConfigSet <name>when your tests require a custom config set.- Debug:
SCOUT_LOG_LEVEL=debug, ornode scripts/playwright test --config <...> --project local --ui
CI enablement
- Scout tests run in CI only for modules listed under
plugins.enabled/packages.enabledin.buildkite/scout_ci_config.yml. node scripts/scout.js generateregisters the module underenabledso the new configs run in CI.
References
Open only what you need:
- Browser authentication helpers and patterns:
references/scout-browser-auth.md - Parallel UI (
spaceTest+scoutSpace) isolation + global setup rules:references/scout-ui-parallelism.md - API services patterns (setup/teardown helpers shared with UI):
../scout-api-testing/references/scout-api-services.md