Neurips Artifact Evaluation
Productivityneurips-artifact-evaluationNeurips Artifact Evaluation is an Agent Skill for productivity published in the brycewang-stanford/Awesome-Journal-Skills repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Use when packaging NeurIPS code, data, models, demos, benchmarks, or other research artifacts for anonymous review, reproducibility, public release, or MLRC-style artifact scrutiny.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/NeurIPS-Skills/skills/neurips-artifact-evaluation/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/neurips-artifact-evaluation, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
NeurIPS Artifact Evaluation
NeurIPS main track does not reduce artifact quality to a generic badge workflow. It expects code, data, and execution details when they are needed to support the scientific claim, and its checklist and code/data guidance make artifact quality visible to reviewers.
Artifact decision
- If the contribution is a method, include training and evaluation code or justify why it cannot be shared.
- If the contribution is a dataset or benchmark, provide metadata, license, preservation plan, representative-use discussion, and access restrictions.
- If the contribution depends on a model, include weights, prompts, decoding settings, compute resources, or a precise explanation of unavailable components.
- If the contribution is theoretical, artifact focus may shift to proof checks, symbolic scripts, experiment notebooks, or counterexample generation.
Anonymous review package
- Keep the ZIP within the current official size limit and anonymize filenames, repository URLs, usernames, commit history, model cards, dataset cards, comments, notebooks, and logs.
- Include a short
READMEwith exact commands, environment, expected runtime, hardware assumptions, and which experiments are reproducible from the package. - Do not require reviewers to run unsafe code outside a secure environment.
- Avoid external links unless the current policy allows them and anonymous browsing is guaranteed.
Public release package
- De-anonymize accepted artifacts.
- Add licenses for code, data, model weights, and generated outputs.
- Archive code in a durable service when appropriate; NeurIPS MLRC guidance recommends Software Heritage for reproducibility papers.
- Keep a mapping from paper claims to commands or notebooks so users can reproduce headline results.
Output format
[Artifact role] method / dataset / benchmark / model / demo / proof / none
[Review package] sufficient / insufficient
[Anonymity risks] <paths, metadata, URLs, usernames>
[Reproducibility gaps] <commands, environment, data, hardware, licenses>
[Public-release plan] <archive, DOI, license, docs>