OpenParable
Skip to content
← Back to skills

Typescript Eval

Testing & Quality
typescript-eval

Typescript Eval is an Agent Skill for testing & quality published in the vellum-ai/vellum-assistant repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Test TypeScript code snippets before persisting as skills

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/vellum-ai/vellum-assistant/blob/HEAD/skills/typescript-eval/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/typescript-eval, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

TypeScript Evaluation

When you need to test a TypeScript snippet before persisting it as a managed skill, use bash directly.

Workflow

1. Write the snippet to a temp file

bash command="mkdir -p /tmp/vellum-eval && cat > /tmp/vellum-eval/snippet.ts << 'SNIPPET_EOF'
<your code here>
SNIPPET_EOF"

2. Run it with bun

bash command="bun run /tmp/vellum-eval/snippet.ts" timeout_seconds=10

3. For function-based testing

If the snippet exports a default or run function, write a runner script:

bash command="cat > /tmp/vellum-eval/runner.ts << 'RUNNER_EOF'
import * as mod from './snippet.ts';
const fn = (mod as any).default ?? (mod as any).run;
const input = {}; // mock input
const result = await fn(input);
console.log(JSON.stringify(result, null, 2));
RUNNER_EOF"

Then run the runner:

bash command="bun run /tmp/vellum-eval/runner.ts" timeout_seconds=10

4. Clean up

bash command="rm -rf /tmp/vellum-eval/"

Guidelines

  • Iteration limit: Max 3 attempts before asking the user for guidance.
  • After successful test: Persist with scaffold_managed_skill only after explicit user consent.
  • Timeout: Use timeout_seconds=10 (or up to 20 for complex snippets).
  • Error handling: Read stdout/stderr from the bash output to diagnose failures.
  • Never persist or delete skills without explicit user confirmation.