OpenParable
Skip to content
← Back to skills

Scout API Testing

Testing & Quality
scout-api-testing

Scout API Testing is an Agent Skill for testing & quality published in the elastic/kibana repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Use when creating, updating, debugging, or reviewing Scout API tests in Kibana (apiTest/apiClient/requestAuth/samlAuth/apiServices), including auth choices, response assertions, and API service patterns.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/elastic/kibana/blob/HEAD/.agents/skills/scout-api-testing/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/scout-api-testing, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Scout API Testing

Core rules (API)

  • API specs live in <module-root>/test/scout*/api/{tests,parallel_tests}/**/*.spec.ts (examples: test/scout/api/..., test/scout_uiam_local/api/...).
  • Use the Scout package that matches the module root:
  • src/platform/** or x-pack/platform/** -> @kbn/scout
  • x-pack/solutions/observability/** -> @kbn/scout-oblt
  • x-pack/solutions/search/** -> @kbn/scout-search
  • x-pack/solutions/security/** -> @kbn/scout-security
  • Prefer a single top-level apiTest.describe(...) per file and avoid nested describe blocks; multiple top-level describes are supported, but files get hard to read quickly.
  • Tags: add { tag: ... } on the suite (or individual tests) so CI/discovery can select the right test target. For solution modules, prefer explicit targets (e.g. [...tags.stateful.classic, ...tags.serverless.observability.complete] in Observability); reserve tags.deploymentAgnostic mainly for platform specs that truly need every deployment-agnostic target (see scout-migrate-from-ftr). Unlike UI tests, API tests don’t currently validate tags at runtime.
  • No @ in test titles: Playwright treats @word in test/describe titles as tags. Do not use @ followed by word characters in titles (e.g., @timestamp, @elastic). Rephrase the title instead (e.g., use timestamp field instead of @timestamp).
  • If the module provides Scout fixtures, import apiTest from <module-root>/test/scout*/api/fixtures to get module-specific extensions. Importing directly from the module’s Scout package is also fine when you don’t need extensions.
  • Browser fixtures are disabled for apiTest (no page, browserAuth, pageObjects).

Imports

  • Test framework + tags: import { apiTest, tags } from '@kbn/scout'; (or the module's Scout package, e.g. @kbn/scout-oblt)
  • Assertions: import { expect } from '@kbn/scout/api'; (or @kbn/scout-oblt/api, etc.) — not from the main entry
  • Types: import type { RoleApiCredentials } from '@kbn/scout';
  • expect is not exported from the main @kbn/scout entry. Use the /api subpath for API tests.

Auth: pick based on endpoint

  • api/* endpoints: use API keys via requestAuth (getApiKey, getApiKeyForCustomRole).
  • internal/* endpoints: use cookies via samlAuth.asInteractiveUser(...).

Recommended test shape

  1. Prepare environment (optional): apiServices/kbnClient/esArchiver in beforeAll.
  2. Authenticate (least privilege): generate credentials in beforeAll and reuse.
  3. Request: call the endpoint with apiClient and the right headers.
  4. Assert: verify statusCode and response body; verify side effects via apiServices/kbnClient when needed.

Important: apiServices/kbnClient run with elevated privileges. Don’t use them to validate the endpoint under test (use apiClient + scoped auth).

Header reminders:

  • State-changing requests usually need kbn-xsrf.
  • Prefer sending x-elastic-internal-origin: kibana for Kibana APIs.
  • Include elastic-api-version for versioned public APIs (e.g. '2023-10-31') or internal APIs (e.g. '1').

Assertions

  • apiClient methods (get, post, put, delete, patch, head) return { statusCode, body, headers }.
  • Use the custom matchers from @kbn/scout/api:
    • expect(response).toHaveStatusCode(200)
    • expect(response).toHaveStatusText('OK')
    • expect(response).toHaveHeaders({ 'content-type': 'application/json' })
  • Standard matchers (toBe, toStrictEqual, toMatchObject, etc.) and asymmetric matchers (expect.objectContaining(...), expect.any(String)) are also available.

API services

  • Put reusable server-side helpers behind apiServices (no UI interactions). Use it for setup/teardown and verifying side effects, not for RBAC validation.
  • Module-local service: create it under <module-root>/test/scout*/api/services/<service>_api_service.ts (or similar). Register it by extending the module's apiServices fixture in <module-root>/test/scout*/api/fixtures/index.ts (prefer { scope: 'worker' } when the helper doesn't need per-test state).
  • Shared service (reused across modules): consider contributing it to the Scout packages under src/platform/packages/shared/kbn-scout/src/playwright/fixtures/scope/worker/apis/.

Extending fixtures

When tests need custom auth helpers or API services, extend apiTest in the module's fixtures/index.ts:

import { apiTest as base } from '@kbn/scout'; // or the module's Scout package
import type { RequestAuthFixture } from '@kbn/scout';

interface MyApiFixtures {
  requestAuth: RequestAuthFixture & { getMyPluginApiKey: () => Promise<RoleApiCredentials> };
}

export const apiTest = base.extend<MyApiFixtures>({
  requestAuth: async ({ requestAuth }, use) => {
    const getMyPluginApiKey = async () =>
      requestAuth.getApiKeyForCustomRole({
        kibana: [{ base: [], feature: { myPlugin: ['all'] }, spaces: ['*'] }],
      });
    await use({ ...requestAuth, getMyPluginApiKey });
  },
});

Tests then import apiTest from the local fixtures: import { apiTest } from '../fixtures';

Parallelism

  • Treat Scout API tests as sequential by default. Parallel API runs require manual isolation (spaces, indices, saved objects) and are uncommon.

Run / debug quickly

  • Use either --config or --testFiles (they are mutually exclusive).
  • Run by config: node scripts/scout.js run-tests --arch stateful --domain classic --config <module-root>/test/scout*/api/playwright.config.ts (or .../api/parallel.playwright.config.ts for parallel API runs)
  • Run by file/dir (Scout derives the right playwright.config.ts vs parallel.playwright.config.ts): node scripts/scout.js run-tests --arch stateful --domain classic --testFiles <module-root>/test/scout*/api/tests/my.spec.ts
  • For faster iteration, start servers once in another terminal: node scripts/scout.js start-server --arch stateful --domain classic [--serverConfigSet <configSet>], then run Playwright directly: node scripts/playwright test --config <...> --project local --grep <tag>.
  • run-tests auto-detects custom config sets from .../test/scout_<name>/... paths.
  • start-server has no Playwright config to inspect, so pass --serverConfigSet <name> when your tests require a custom config set.
  • Debug: SCOUT_LOG_LEVEL=debug

CI enablement

  • Scout tests run in CI only for modules listed under plugins.enabled / packages.enabled in .buildkite/scout_ci_config.yml.
  • node scripts/scout.js generate registers the module under enabled so the new configs run in CI.

References

Open only what you need:

  • requestAuth vs samlAuth, headers, and least-privilege auth tips: references/scout-api-auth.md
  • Creating and registering apiServices helpers (kbnClient + retries + logging): references/scout-api-services.md