Validate Oas
Testing & Qualityvalidate-oasValidate Oas is an Agent Skill for testing & quality published in the elastic/kibana repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Use when you need a quick VALID or NOT VALID result for a scoped Kibana OAS area, and first ensure the generated `oas_docs` inputs are up to date so validation runs against the current environment rather than stale snapshots.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/elastic/kibana/blob/HEAD/.agents/skills/validate-oas/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/validate-oas, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Validate OAS
Overview
Use node ./scripts/validate_oas_docs.js for a fast pass/fail check of a specific API area.
This skill is intentionally minimal:
- Return only
VALIDorNOT VALID. - If
NOT VALID, mention thedebug-oasskill for detailed issue debugging. - Use this skill first for quick pass/fail.
- Hand off to
debug-oaswhen the developer wants issue categorization or examples.
Before validating, make sure the generated OAS artifacts in oas_docs are current. Treat stale generated files as an environment/setup problem, not a validation result.
Environment setup
Refresh the generated OAS inputs before validation when:
- the developer asks for a fresh or CI-like validation run
oas_docsmay be stale after switching branches or pulling changes- scoped validation gives surprising results that may come from outdated generated files
Do not refresh first when:
- the developer explicitly wants a fast local re-check only
- the developer already refreshed
oas_docsin this session and no relevant inputs changed since then
Refresh flow:
- Bootstrap dependencies if needed:
yarn kbn bootstrap
- Regenerate captured OAS snapshots using the same include paths as Buildkite:
node scripts/capture_oas_snapshot \
--include-path /api/status \
--include-path /api/alerting/rule/ \
--include-path /api/alerting/rules \
--include-path /api/actions \
--include-path /api/security/role \
--include-path /api/spaces \
--include-path /api/streams \
--include-path /api/fleet \
--include-path /api/saved_objects \
--include-path /api/maintenance_window \
--include-path /api/agent_builder \
--include-path /api/workflows \
--include-path /api/security/entity_store \
--include-path /api/dashboards \
--include-path /api/visualizations
- Run the OpenAPI bundling scripts:
bash .buildkite/scripts/steps/openapi_bundling/security_solution_openapi_bundling.sh
bash .buildkite/scripts/steps/openapi_bundling/final_merge.sh
- Rebuild the final OAS documents:
cd oas_docs && make api-docs
After this refresh flow completes, run the scoped validation command.
Source of truth:
- Keep the
capture_oas_snapshotinclude-path list aligned with.buildkite/scripts/steps/checks/capture_oas_snapshot.sh. - Keep the openapi_bundling step list aligned with
.buildkite/scripts/steps/openapi_bundling/. - If the Buildkite command changes, update this skill to match it.
Required interaction flow
- Ask what APIs the developer is working on.
- Ask for one or more HTTP API paths (for example
/api/fleet/agent_policies). - Ensure
oas_docsis up to date first when needed by following the environment setup flow above. - Run validation using those route-style
--pathfilters with--only traditionalby default and always include--skip-printing-issues. - Return only:
VALID, orNOT VALID(and mentiondebug-oasfor details).
Do not skip questions (1) and (2) unless the developer already provided the API paths.
Path format
Use normal route-style API paths for --path:
/api/fleet/agent_policies/api/fleet/agent_policies/{agentPolicyId}
Do not manually convert to JSON pointers. The CLI handles conversion for error filtering internally.
Commands
Environment refresh:
yarn kbn bootstrap
node scripts/capture_oas_snapshot \
--include-path /api/status \
--include-path /api/alerting/rule/ \
--include-path /api/alerting/rules \
--include-path /api/actions \
--include-path /api/security/role \
--include-path /api/spaces \
--include-path /api/streams \
--include-path /api/fleet \
--include-path /api/saved_objects \
--include-path /api/maintenance_window \
--include-path /api/agent_builder \
--include-path /api/workflows \
--include-path /api/security/entity_store \
--include-path /api/dashboards \
--include-path /api/visualizations
bash .buildkite/scripts/steps/openapi_bundling/security_solution_openapi_bundling.sh
bash .buildkite/scripts/steps/openapi_bundling/final_merge.sh
cd oas_docs && make api-docs
Default scoped validation:
node ./scripts/validate_oas_docs.js --only traditional --skip-printing-issues --path <api_route_prefix>
Multiple path filters are supported:
node ./scripts/validate_oas_docs.js --only traditional \
--skip-printing-issues \
--path /api/fleet/agent_policies \
--path /api/fleet/agent_policies/{agentPolicyId}
Optional (only when explicitly requested):
node ./scripts/validate_oas_docs.js --only serverless --skip-printing-issues --path <api_route_prefix>
Default scoping note:
- Prefer
--only traditionalby default because it matches the common local debugging path and keeps output narrower. - Use
--only serverlessonly when the developer explicitly asks for it.
Result rules
- Determine status from CLI output:
Found 0 errors in ...->VALIDFound N errors in ...whereN > 0->NOT VALID
- If validation was run without first refreshing obviously stale
oas_docs, note that the result may reflect outdated generated inputs and refresh before concluding the spec is clean or broken. - If CLI output indicates no matched paths (for example
None of the provided --path filters matched any content), keep status asVALIDbut include an explicit warning. - Output must stay concise:
VALIDVALID (WARNING: no actual paths were matched.)NOT VALID. Use the debug-oas skill for detailed issues.
Output template
Use exactly one of these responses:
VALID
VALID (WARNING: no actual paths were matched.)
NOT VALID. Use the debug-oas skill for detailed issues.