Running Smoke Tests
Testing & Qualityrunning-smoke-testsRunning Smoke Tests is an Agent Skill for testing & quality published in the jeremylongshore/claude-code-plugins-plus-skills repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Execute fast smoke tests validating critical functionality after deployment. Use when performing specialized testing. Trigger with phrases like "run smoke tests", "quick validation", or "test critical paths".
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/jeremylongshore/claude-code-plugins-plus-skills/blob/HEAD/plugins/testing/smoke-test-runner/skills/running-smoke-tests/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/running-smoke-tests, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Smoke Test Runner
Overview
Execute fast, high-confidence smoke tests that validate critical application functionality after deployment or build. Smoke tests verify that the application starts, core user flows work, and key integrations respond -- without running the full test suite.
Prerequisites
- Application deployed and accessible at a known URL or running locally
- HTTP client available (
curl,wget,node-fetch, or Playwright) - List of critical endpoints and user flows to validate
- Expected response codes and content patterns for each check
- CI/CD pipeline hook for post-deployment validation
Instructions
- Identify the critical paths that constitute a "working" application:
- Health check endpoint returns 200 with expected body.
- Homepage loads and contains key UI elements.
- Authentication flow succeeds with test credentials.
- Primary API endpoint returns valid data.
- Database connection is active and responding.
- Create a smoke test configuration listing each check:
- URL or command to execute.
- Expected HTTP status code (200, 301, etc.).
- Response body pattern to match (substring or regex).
- Maximum acceptable response time (e.g., 3 seconds).
- Write the smoke test suite as a lightweight script or test file:
- Use
curlfor HTTP checks or Playwright for browser-based checks. - Run checks sequentially for simplicity (parallel for speed if independent).
- Fail fast on the first critical failure.
- Log each check result with pass/fail, response time, and status code.
- Use
- Implement timeout guards:
- Set a global timeout of 60 seconds for the entire smoke suite.
- Set per-check timeouts of 5-10 seconds.
- Treat timeouts as failures, not retries.
- Add deployment-gate integration:
- On success: proceed with deployment promotion or traffic shifting.
- On failure: trigger rollback and send alert notification.
- Report results to CI/CD dashboard and Slack/Teams webhook.
- Store smoke test results as CI artifacts for audit trail.
- Schedule periodic smoke runs (every 5 minutes in production) as synthetic monitoring.
Output
- Smoke test script (
scripts/smoke-test.shortests/smoke.test.ts) - Pass/fail result for each critical check with response times
- Deployment gate verdict (PASS or FAIL with reason)
- CI artifact with timestamped smoke test log
- Alert payload for failed checks (Slack webhook, PagerDuty, etc.)
Error Handling
| Error | Cause | Solution |
|---|---|---|
| Connection refused | Application not yet ready after deployment | Add a startup wait with exponential backoff (max 30 seconds) before running smoke tests |
| 503 Service Unavailable | Application is starting or behind a load balancer draining | Retry with 2-second delay up to 3 times; check load balancer health check status |
| Unexpected redirect (301/302) | URL changed or SSL redirect not accounted for | Follow redirects with curl -L; update expected URLs in smoke config |
| Content mismatch | Page content changed but smoke test pattern is too specific | Use broad patterns (check for <title> or key element IDs, not exact text) |
| Timeout on database check | Database migration running or connection pool exhausted | Increase timeout for database checks; verify migration completed before smoke tests |
Examples
Shell-based smoke test script:
#!/bin/bash
set -e
BASE_URL="${1:-http://localhost:3000}" # 3000: 3 seconds in ms
PASS=0; FAIL=0
check() {
local name="$1" url="$2" expected="$3"
status=$(curl -s -o /dev/null -w '%{http_code}' --max-time 5 "$url")
if [ "$status" = "$expected" ]; then
echo "PASS: $name (HTTP $status)"
((PASS++))
else
echo "FAIL: $name (expected $expected, got $status)"
((FAIL++))
fi
}
check "Health check" "$BASE_URL/health" "200" # HTTP 200 OK
check "Homepage" "$BASE_URL/" "200" # HTTP 200 OK
check "API status" "$BASE_URL/api/status" "200" # HTTP 200 OK
check "Login page" "$BASE_URL/login" "200" # HTTP 200 OK
echo "Results: $PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ] || exit 1
Playwright smoke test:
import { test, expect } from '@playwright/test';
test('homepage loads with navigation', async ({ page }) => {
await page.goto('/', { timeout: 10000 }); # 10000: 10 seconds in ms
await expect(page.locator('nav')).toBeVisible();
await expect(page).toHaveTitle(/My App/);
});
test('API health endpoint responds', async ({ request }) => {
const response = await request.get('/api/health');
expect(response.ok()).toBeTruthy();
expect(await response.json()).toHaveProperty('status', 'ok');
});
Resources
- Smoke testing methodology: https://martinfowler.com/bliki/SmokeTest.html
- Playwright API testing: https://playwright.dev/docs/api-testing
- curl documentation: https://curl.se/docs/manpage.html
- GitHub Actions deployment gates: https://docs.github.com/en/actions/deployment/targeting-different-environments/using-environments-for-deployment