Typescript Eval
Testing & Qualitytypescript-evalTypescript Eval is an Agent Skill for testing & quality published in the vellum-ai/vellum-assistant repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Test TypeScript code snippets before persisting as skills
QUICK START
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/vellum-ai/vellum-assistant/blob/HEAD/skills/typescript-eval/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/typescript-eval, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
TypeScript Evaluation
When you need to test a TypeScript snippet before persisting it as a managed skill, use bash directly.
Workflow
1. Write the snippet to a temp file
bash command="mkdir -p /tmp/vellum-eval && cat > /tmp/vellum-eval/snippet.ts << 'SNIPPET_EOF'
<your code here>
SNIPPET_EOF"
2. Run it with bun
bash command="bun run /tmp/vellum-eval/snippet.ts" timeout_seconds=10
3. For function-based testing
If the snippet exports a default or run function, write a runner script:
bash command="cat > /tmp/vellum-eval/runner.ts << 'RUNNER_EOF'
import * as mod from './snippet.ts';
const fn = (mod as any).default ?? (mod as any).run;
const input = {}; // mock input
const result = await fn(input);
console.log(JSON.stringify(result, null, 2));
RUNNER_EOF"
Then run the runner:
bash command="bun run /tmp/vellum-eval/runner.ts" timeout_seconds=10
4. Clean up
bash command="rm -rf /tmp/vellum-eval/"
Guidelines
- Iteration limit: Max 3 attempts before asking the user for guidance.
- After successful test: Persist with
scaffold_managed_skillonly after explicit user consent. - Timeout: Use
timeout_seconds=10(or up to 20 for complex snippets). - Error handling: Read stdout/stderr from the bash output to diagnose failures.
- Never persist or delete skills without explicit user confirmation.