Kornia Developer
Developmentkornia-developerKornia Developer is an Agent Skill for development published in the kornia/kornia repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Use when developing on kornia — making an op torch.compile / dynamo compatible, fixing a graph break, or optimizing for speed. Codifies the compile-first workflow — genuine fullgraph fixes (no is_compiling hacks), byte-to-byte eager preservation, cross-library benchmarking toward the 10x-vs-albumentations moonshot.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/kornia/kornia/blob/HEAD/.claude/skills/kornia-developer/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/kornia-developer, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
kornia-developer: compile-first optimization
Make a kornia function genuinely torch.compile(fullgraph=True)-compatible and measurably faster. This is a rigid workflow — follow every step.
First principle: foundation, not green tests
The goal is a function that is genuinely fullgraph and genuinely faster — not a test that happens to pass. A green test is evidence, never the objective. Every step below exists to make the underlying claim true; if you find yourself tuning a test to go green without understanding why the code now works, stop — you are building on sand. Concrete traps this workflow has hit:
- A test can pass while never exercising the claim. Compiling
op(x, params=pregenerated_params)skipsforward_parameters, so a "fullgraph" dynamo test that passes pre-generated params never traced the parameter-generation path. Test the real user path (op(x)), not a shortcut that dodges the code you're claiming to fix. - "Works on my machine" is not "works."
int(tensor)constant-folds under dynamo on some torch versions (e.g. 2.10) and lowers to a graph-breaking.item()on others (e.g. 2.9.1). A fullgraph claim verified only on the local torch is worthless if CI runs a different one. Verify on the CI torch version (the PR dynamo job is the source of truth), and prefer fixes that are version-independent by construction — build tensors from tensors (torch.stackof scalar0/1), neverint(tensor)ortorch.tensor([[... python int ...]]). - Understand why a branch is dead before deleting it.
tensor == torch.Size(...)doesn't broadcast — it falls back to identity==and is silently alwaysFalse. Code can "work" with a branch that never runs. Diagnose the mechanism; don't pattern-match a fix.
The non-negotiable rule: no is_compiling() hacks
torch.compiler.is_compiling() guards are banned as the fix. They make the compiled path skip the data-dependent code instead of making it traceable, so eager and compile run different paths and checks are silently dropped under compile. That is "fullgraph by exclusion," not genuine fullgraph — and it leaves the op only partially compiled, killing the speedup.
The fix must run the same code path in eager and compile.
Workflow (create a TodoWrite item per step)
-
Reproduce the break.
torch.compile(fn, fullgraph=True)(x)and capture the exact graph-break location (a traceback frame inkornia/…). Never guess. -
Classify the break and apply the genuine fix:
Break Genuine fix (single path) if (t < 0).any(): raise— value validationtorch._assert_async((t >= 0).all(), "msg")— keeps the check, no breakbits==0/==8style logic branch on a tensorbranchless torch.where(cond, a, b)— compute both, selectif not mask.any(): break— early-exit in a fixed-bound loopdelete it; trailing iterations must be provable no-ops if mask.all(): return None— early-out returning a different typealways return the value; make the consumer's use a no-op if batch_prob.sum()==1— gate on a {0,1} tensorbranchless multiply ( gate * value)Python if p == 1wherepis a Python float/boolleave it — resolved at trace time, not a break If none apply (unbounded
while,.item()-driven shapes, random-permutation dispatch, dynamic-shapenonzero/unique), it needs a redesign or a maintainer decision — do not force a hack. Document it and stop. -
Verify genuine fullgraph == eager, and byte-to-byte eager preservation. Two separate checks:
- Compiled matches eager on the same path:
torch.allclose(torch.compile(fn, fullgraph=True)(x), fn(x), atol=1e-5). - The fix must not change eager output at all. Save the op's output on
mainand on your branch under the same seed and assert byte-identical —torch.equal(old, new), notallclose. This is the contract: a compile refactor is only acceptable if existing users get exactly the same numbers. If it can't be byte-identical (e.g. a genuinely new opt-in mode), gate the change behind a new argument and leave the default path byte-identical.
# on main: torch.manual_seed(0); torch.save(fn(x), "old.pt") # on branch: torch.manual_seed(0); assert torch.equal(fn(x), torch.load("old.pt", weights_only=True))For branchless rewrites of edge-cased logic, verify exhaustively across the edge inputs (e.g. every
bits0..8), not one sample. For anything touching RNG (augmentation base, generators), a shifted draw order breaks byte-identity — check the whole module suite passes unchanged as corroboration. - Compiled matches eager on the same path:
-
Benchmark before/after — REQUIRED for every touched function. A compile fix that doesn't speed anything up (or regresses eager) must be justified. Benchmarking is an experiment, not a vibe — hold it to experimental standards:
import torch.utils.benchmark as bench def us(f,*a): return bench.Timer(stmt="f(*a)",globals={"f":f,"a":a}).blocked_autorange(min_run_time=1.0).median*1e6 eager = us(fn, *args) c = torch.compile(fn, fullgraph=True); c(*args) # warmup — compile + allocator + cudnn autotune all happen on the first call comp = us(c, *args)Methodology that makes the number trustworthy — deviate and the comparison is noise:
- Warm up before timing (first call pays compilation / autotune / lazy-init). Never time a single call — use
blocked_autorange(statistical, median of many) so you report signal, not scheduler jitter. - Same machine, same process, back-to-back for before/after. Never compare a number from one box to a number from another.
torch.utils.benchmarkhandles CUDA synchronization; a hand-rolledtime.time()around a CUDA call measures launch latency, not work. - Realistic shape + batch (e.g.
(32,3,256,256)for augmentation) — a(1,3,8,8)toy inflates Python overhead and hides kernel cost. Record hardware, torch version, commit, date with the numbers (edge silicon like Jetson is directional; headline GPU-leadership claims need a datacenter GPU). - Confirm the rewrite didn't regress eager (branchless "compute both branches" and deleted early-exits add eager work — measure it). Put the before/after table in the PR.
Benchmark against every other library — kornia's numbers are meaningless in isolation. Use the harnesses, don't hand-roll:
benchmarks/augmentation/all_libraries.py(kornia eager+compiled vs torchvision v2, albumentations, OpenCV, PIL, kornia-rs),cross_library.py(focused three-way),pipeline.py(end-to-end). Readbenchmarks/augmentation/README.mdfirst — it documents the regimes and the standing improvement list, and it is where durable results and the honest interpretation live (update it when you move a number). Read the columns honestly and state the regime: OpenCV / kornia-rs win CPU/uint8/single-image, torchvision v2 wins raw float-tensor throughput, kornia's regime is GPU-batched + differentiable + compiled — the only one where differentiable, on-device augmentation exists at all.The moonshot is 10× vs albumentations for the augmentation package — frame every perf fix against that target. Today kornia trails on CPU; the levers that close then invert the gap: (a)
torch.compile(the fix you just made — ~2–3×), (b) a uint8 fast path (albumentations' whole edge is uint8 + OpenCV; kornia upcasts to float32), (c) pushing the hot op intokornia-rs(the Rust backend albumentations can't match). A compile fix that only reaches parity is a step toward 10×, not the destination — say in the PR which lever is still on the table. - Warm up before timing (first call pays compilation / autotune / lazy-init). Never time a single call — use
-
Lock it in. Add/confirm a
test_dynamofor the op that exercises the real path — compile the full forward (torch.compile(op, fullgraph=True)(x)with no pre-generatedparams), not justapply_transformwith params fed in, so parameter generation is covered too. A PR-timedynamoCI job now runs the compile-clean core underinductoron the CI torch version (.github/workflows/pr_test_cpu.yml), so a fullgraph regression is caught at PR time on the real torch — but locally you must still run withKORNIA_TEST_OPTIMIZER=inductor(the default matrix sets it empty and deselects these tests). If you add an op to a scoped-core dir, the CI job will exercise it; confirm it passes there, not only on your local torch. -
Verify the suite is unchanged. Run the op's full test file with
--dtype=float32. For shared helpers or the augmentation base, run the whole module suite — a branchless rewrite can shift RNG consumption (e.g. always drawing a sample that used to be conditional); the suite passing unchanged is the proof no seeded test depends on it. -
PR. One op (or one shared helper) per PR. Body: the break, the genuine fix, the fullgraph==eager evidence, and the benchmark table.
ruff check+ruff format --checkclean (pinned version from.pre-commit-config.yaml).
Leverage: prefer shared fixes
The biggest wins are shared helpers/base classes, where one genuine fix unblocks many ops:
kornia/augmentation/base.py__batch_prob_generator__gate → unblocked ~13 augmentations.kornia/losses/_utils.pymask_ignore_pixels→ unblocked dice/focal/tversky.
When an op breaks, trace to the deepest kornia/ frame first — the break is often in a shared utility, and fixing it there is far higher leverage than per-op.
What's already compile-clean
Most of the numeric core already compiles (filters, color, geometry transforms, losses, morphology, metrics). Sweep before assuming a break exists.