Dataset Profiler
Documentsdataset-profilerDataset Profiler is an Agent Skill for documents published in the oxbshw/LLM-Agents-Ecosystem-Handbook repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Use when first encountering a new dataset — produces a structured profile (schema, missingness, distributions, outliers, gotchas) before any analysis.
QUICK START
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/oxbshw/LLM-Agents-Ecosystem-Handbook/blob/HEAD/skills/catalog/dataset-profiler/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/dataset-profiler, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Dataset Profiler
When to use
- A new dataset arrives and you need to understand it before using it
- Before reproducing an analysis that referenced a dataset
- When data quality is suspect ("the chart looked wrong")
When NOT to use
- Streaming / online data (this is point-in-time)
- Sensitive PII without an explicit allow-list
Inputs
| Name | Type | Required | Notes |
|---|---|---|---|
path | path | yes | CSV / Parquet / JSONL |
target | string | no | column of interest (gets extra distribution detail) |
Outputs
profile.md with: Source, Schema, Missingness, Distributions, Outliers, Joins / keys, Gotchas, Open questions.
Workflow
- Load with the right reader (extension-detected); record row count, file size
- Schema: column → dtype → nullable → example value
- Missingness: % per column, top columns by missingness
- Distributions: numeric (min, p50, p95, max, std), categorical (top-k, cardinality)
- Outliers: flag rows beyond p99 + 3·IQR for numerics
- Identify potential keys (unique columns) and join candidates
- Gotchas: timezone columns, mixed encodings, suspicious all-zero rows, magic values (
-1,9999-12-31) - Open questions: ambiguous columns / values that need owner input
References
Success criteria
- Every column appears in Schema + Missingness
- Outliers section includes example rows
- Gotchas section is non-empty (real datasets always have some)
Failure modes
- File too large to read in memory → switch to streaming + sampled stats; flag prominently
- Encoding fails → try common alternatives; if all fail, surface and stop