OpenParable
Skip to content
← Back to skills

Sitk Upload Binary Data

Testing & Quality
sitk-upload-binary-data

Sitk Upload Binary Data is an Agent Skill for testing & quality published in the SimpleITK/SimpleITK repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Upload binary testing data to SimpleITK ExternalData repository. Use when: adding test data, uploading binary file, creating content link, new test input, ExternalData upload, sha512 content link.

View on GitHub ↗
QUICK START

How to use this skill

Bring this guide into your coding agent with a prompt tailored to the tool you use.

  1. Open your project in Codex.
  2. Copy and paste the prompt below.
  3. Review files and risks before approving any action.
Prompt to paste
Inspect the Agent Skill at https://github.com/SimpleITK/SimpleITK/blob/HEAD/.github/skills/sitk-upload-binary-data/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/sitk-upload-binary-data, but wait for my approval before writing files or running scripts.

Copying this prompt does not install or run the skill. Review third-party files before use.

Upload Binary Data to SimpleITK ExternalData

Upload binary files (test inputs, baseline images, etc.) to the SimpleITK/SimpleITKExternalData GitHub repository by computing their SHA-512 hash, placing them in the .ExternalData object store, creating content link files, and opening a draft PR.

When to Use

  • Adding new test input images or baseline outputs
  • Uploading any binary file that should not be committed to the SimpleITK Git repo
  • Replaces input file with a .sha512 CMake ExternalData content link

Required Tools

Verify these are available before proceeding:

ToolPurposeCheck command
gitRepository operationsgit --version
ghGitHub CLI for forking/PRsgh auth status
sha512sum or shasumCompute file hashessha512sum --version (Linux) or shasum --version (macOS)

Background

Binary test data in SimpleITK uses CMake ExternalData. Content link files (e.g. Testing/Data/Input/image.nrrd.sha512) contain only a 128-character lowercase hex SHA-512 hash followed by a newline. The .ExternalData directory at the SimpleITK source root is a clone of SimpleITK/SimpleITKExternalData and also a local CMake ExternalData object store (listed in .gitignore), so builds work immediately once a file is placed there.

Always create .sha512 content links and SHA512/<hash> objects. See resources/external-data-background.md for full history, legacy considerations, and object store/URL resolution details.

Bundled Script

scripts/hash_and_stage.sh

A bash script that handles hashing, copying, content-link creation, and cleanup for one or more binary files. Run from the SimpleITK source root.

Usage:

bash .github/skills/sitk-upload-binary-data/scripts/hash_and_stage.sh <file1> [<file2> ...]

What it does:

  1. Validates prerequisites (sha512sum, git repo, .ExternalData clone).
  2. For each file:
    • Computes the SHA-512 hash via sha512sum.
    • Copies the binary to .ExternalData/SHA512/<hash> (skips if the hash already exists).
    • Writes <file>.sha512 containing the 128-char hex hash + newline.
    • Removes the original binary file.
    • Notes any existing .md5 content link for removal.
  3. Prints suggested git add / git rm commands.

Output labels:

  • COPY — binary copied to object store
  • SKIP — hash already present, no copy needed
  • LINK — .sha512 content link created
  • DEL — original binary removed

Does NOT: create branches, commit, push, or open PRs — those steps are performed separately (see Step 4 below).

Exit codes: non-zero on any error (missing file, bad .ExternalData directory, missing sha512sum). The error message explains the problem.

Procedure

Given one or more binary file paths relative to the SimpleITK source root. Example: Testing/Data/Baseline/BasicFilters_NormalizedCorrelationImageFilter_NoMask.nrrd

Step 1 — Verify the file exists

Confirm each binary file is present at the given path. If not, stop and tell the user.

Step 2 — Verify the .ExternalData clone and select a push remote

All commands run from the SimpleITK source root. First confirm .ExternalData is a real git clone:

test -d .ExternalData/.git

If this fails, stop and tell the user:

The .ExternalData directory must be a clone of SimpleITK/SimpleITKExternalData. To set it up, run:

gh repo clone SimpleITK/SimpleITKExternalData .ExternalData

Otherwise, list its remotes to identify both the upstream and the remote to push to:

git -C .ExternalData remote -v
  • The remote pointing at SimpleITK/SimpleITKExternalData itself (typically origin/upstream) is <upstream-remote> — used in Step 4 to fetch and base the branch on main. Do not assume it is named origin; read the actual name from the remote -v output.
  • Do not run gh repo fork — never create a fork automatically. Among the other remotes (not <upstream-remote>):
    • If exactly one remains, use its name as <push-remote>.
    • Otherwise (zero or multiple candidates), ask the user which remote name to use as <push-remote>, listing whatever remote names were found. If none exist, tell the user they need to add one first (e.g. by running gh repo fork --remote --remote-name <name> themselves).

Use <upstream-remote> and <push-remote> in Step 4.

Step 3 — Hash, copy, create content links, remove originals

Run the bundled script from the SimpleITK source root:

bash .github/skills/sitk-upload-binary-data/scripts/hash_and_stage.sh \
  path/to/file1 [path/to/file2 ...]

Capture the hash from the script output (the COPY or SKIP line) or read it from the generated .sha512 file:

HASH=$(cat path/to/file.sha512)

If the script fails, read its error output and report to the user.

Step 4 — Create branch, commit, and PR in ExternalData repo

All commands use git -C .ExternalData to stay in the source root. Run from the SimpleITK source root:

git -C .ExternalData fetch <upstream-remote>
git -C .ExternalData checkout -B <branch-name> <upstream-remote>/main
git -C .ExternalData add SHA512/<hash>
git -C .ExternalData commit -m "Add <original-filename>"
git -C .ExternalData push --force-with-lease <push-remote> HEAD:<branch-name>

The gh CLI does not support -C, so use a subshell for PR creation. The subshell does not change the parent shell's working directory:

(cd .ExternalData && gh pr create --repo SimpleITK/SimpleITKExternalData \
  --title "Add <original-filename>" \
  --body "Binary testing data for SimpleITK." \
  --draft)

Branch naming: Use the sanitized original filename (replace dots and special chars with hyphens, no prefix). For multiple files use upload-N-files.

Commit message: Add filename1, filename2, ... — include original filenames so reviewers know what the data is.

Step 5 — Stage the content link in SimpleITK

Back in the SimpleITK source root:

git add -- path/to/file.sha512

Step 6 — Report to user

Summarize:

  • Content link created at <path>.sha512
  • Data placed in .ExternalData/SHA512/<hash> (builds work locally now)
  • Draft PR opened on SimpleITK/SimpleITKExternalData (include URL)
  • Remind: data won't be available via S3/CMake download until the ExternalData PR is merged

Troubleshooting Resources