# Neurips Artifact Evaluation

> Neurips Artifact Evaluation is an Agent Skill for productivity published in the brycewang-stanford/Awesome-Journal-Skills repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Use when packaging NeurIPS code, data, models, demos, benchmarks, or other research artifacts for anonymous review, reproducibility, public release, or MLRC-style artifact scrutiny.

- Skill name: `neurips-artifact-evaluation`
- Category: [Productivity](https://www.openparable.com/skill/productivity/1)
- Source: https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/NeurIPS-Skills/skills/neurips-artifact-evaluation/SKILL.md
- Repository: brycewang-stanford/Awesome-Journal-Skills (805 GitHub stars when collected)

## SKILL.md

# NeurIPS Artifact Evaluation

NeurIPS main track does not reduce artifact quality to a generic badge workflow. It expects code,
data, and execution details when they are needed to support the scientific claim, and its checklist
and code/data guidance make artifact quality visible to reviewers.

## Artifact decision

- If the contribution is a method, include training and evaluation code or justify why it cannot be
  shared.
- If the contribution is a dataset or benchmark, provide metadata, license, preservation plan,
  representative-use discussion, and access restrictions.
- If the contribution depends on a model, include weights, prompts, decoding settings, compute
  resources, or a precise explanation of unavailable components.
- If the contribution is theoretical, artifact focus may shift to proof checks, symbolic scripts,
  experiment notebooks, or counterexample generation.

## Anonymous review package

- Keep the ZIP within the current official size limit and anonymize filenames, repository URLs,
  usernames, commit history, model cards, dataset cards, comments, notebooks, and logs.
- Include a short `README` with exact commands, environment, expected runtime, hardware assumptions,
  and which experiments are reproducible from the package.
- Do not require reviewers to run unsafe code outside a secure environment.
- Avoid external links unless the current policy allows them and anonymous browsing is guaranteed.

## Public release package

- De-anonymize accepted artifacts.
- Add licenses for code, data, model weights, and generated outputs.
- Archive code in a durable service when appropriate; NeurIPS MLRC guidance recommends Software
  Heritage for reproducibility papers.
- Keep a mapping from paper claims to commands or notebooks so users can reproduce headline results.

## Output format

```text
[Artifact role] method / dataset / benchmark / model / demo / proof / none
[Review package] sufficient / insufficient
[Anonymity risks] <paths, metadata, URLs, usernames>
[Reproducibility gaps] <commands, environment, data, hardware, licenses>
[Public-release plan] <archive, DOI, license, docs>
```

---

Source: https://www.openparable.com/skill/productivity/neurips-artifact-evaluation (OpenParable). Cite the canonical URL above.
