# Acmmm Reproducibility

> Acmmm Reproducibility is an Agent Skill for development published in the brycewang-stanford/Awesome-Journal-Skills repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Use when strengthening the reproducibility of an ACM MM (ACM Multimedia) paper or preparing for the ACM MM Reproducibility track and ACM artifact badging — capturing environments, media/data access, seeds, and multimodal pipelines so an independent reviewer can rebuild results and reach Artifacts Evaluated or Results Reproduced badges.

- Skill name: `acmmm-reproducibility`
- Category: [Development](https://www.openparable.com/skill/development/1)
- Source: https://github.com/brycewang-stanford/Awesome-Journal-Skills/blob/HEAD/ACM-MM-Skills/skills/acmmm-reproducibility/SKILL.md
- Repository: brycewang-stanford/Awesome-Journal-Skills (805 GitHub stars when collected)

## SKILL.md

# ACM MM Reproducibility

Use this to make an ACM Multimedia result rebuildable — both for main-track credibility and
for the dedicated **Reproducibility track**, which routes artifacts through ACM's
badging pipeline. Multimedia adds a wrinkle: the data is often video, audio, or interactive
media, and "run the code" is not enough if a reviewer cannot obtain or render the media.

## What reproducibility means here

ACM's artifact model distinguishes availability, evaluation, and reproduction. Map your goal
to the badge you are actually pursuing:

| Badge (ACM terminology) | What it asserts | What you must ship |
|---|---|---|
| Artifacts Available | The artifact is publicly, permanently retrievable | A DOI/archived repository with the code and media pointers |
| Artifacts Evaluated (Functional/Reusable) | Reviewers ran it and it works / is reusable | Build + run instructions, environment, documentation |
| Results Reproduced | An independent team reproduced the paper's results | A pipeline that regenerates the reported numbers/media |

Confirm the exact badge set offered for the current cycle on the Reproducibility-track call;
ACM's badge names and criteria evolve.

## The multimodal reproducibility ledger

Keep a single record that ties each reported result to the code, data, and config that
produced it:

```text
result: Table 2, row "full model"
  code commit: <hash>
  config: configs/full.yaml
  data: <dataset name + version + anonymous mirror for review>
  media preprocessing: <fps, sample rate, caption source>
  seed(s): <list>
  hardware: <GPU/CPU, hours>
  expected output: results/table2_full.json
```

## Media and data access

- Provide an **anonymous, working** path to the data during double-blind review — a mirror
  that a reviewer can actually download, not a placeholder.
- State the **license** and any consent/usage terms; user-generated media often cannot be
  redistributed, so document how a reviewer obtains it.
- Pin **preprocessing**: frame rate, resampling, transcription source, and alignment — small
  differences here silently break multimodal results.

## Determinism where it is achievable

- Fix and log seeds; note where nondeterminism is irreducible (e.g., some GPU kernels) and
  report variance instead of pretending to bit-exactness.
- Version the environment (container or lockfile) and record hardware, since media models are
  often memory- and throughput-sensitive.

## Reproducibility-track readiness pass

- The Reproducibility and Open Source tracks are **single-blind**, so the artifact carries
  its real identity — but the *main-track* review artifact must still be anonymous.
- Package for a stranger: a reviewer with your README and nothing else should build, run, and
  hit an expected-output check within a bounded time.
- Include a smoke check (see [`../../resources/code/README.md`](../../resources/code/README.md))
  that verifies structure and media rendering before you submit.

## Where multimodal pipelines silently break

Multimedia reproduction fails in places pure-code reproduction does not:

- **Codec and container drift** — a video re-encoded with a different codec changes pixel values
  and breaks frame-exact results; pin the decode path.
- **Sample-rate and resampling** — audio resampled by a different library shifts features; record
  the exact resampler and rate.
- **Caption/transcript source** — if captions come from an ASR system or a platform, name the
  version; a different transcript is a different input.
- **Frame sampling** — "every k-th frame" depends on the container's frame rate; state fps and
  the sampling rule.

A reproduction package that omits these looks complete but regenerates different numbers, which
is worse than an honest gap.

## Anonymous review vs. public artifact

The review artifact and the release artifact have different rules, and conflating them causes
anonymity leaks or dead links:

- **During review** (double-blind tracks): anonymous repository, anonymous data mirror, no author
  names in code comments, media metadata, or commit history.
- **At release** (camera-ready): the public, de-anonymized repository with a permanent archive
  (DOI), the license, and the final media — replacing, not merely supplementing, the anonymous
  mirror.

## Output format

```text
[Badge target] Available / Evaluated / Results Reproduced
[Ledger] complete / gaps: <which results lack a trace>
[Data access] anonymous + licensed / broken or unlicensed
[Media preprocessing] pinned / underspecified
[Determinism] seeds+env logged / gaps
[Track blinding] correct for chosen track / mismatch
[Top fixes] <ordered>
```

---

Source: https://www.openparable.com/skill/development/acmmm-reproducibility (OpenParable). Cite the canonical URL above.
