Diffusers Code
Developmentdiffusers-codeDiffusers Code is an Agent Skill for development published in the vladmandic/sdnext repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.
Create or edit code that is compliant with Hugging Face diffusers conventions, including models, pipelines, schedulers, tests, docs, and PR preparation targeting diffusers.
How to use this skill
Bring this guide into your coding agent with a prompt tailored to the tool you use.
- Open your project in Codex.
- Copy and paste the prompt below.
- Review files and risks before approving any action.
Inspect the Agent Skill at https://github.com/vladmandic/sdnext/blob/HEAD/.github/skills/diffusers-code/SKILL.md and all supporting files. Report provenance, permissions, scripts, dependencies, and security risks. Propose an exact installation into .agents/skills/diffusers-code, but wait for my approval before writing files or running scripts.
Copying this prompt does not install or run the skill. Review third-party files before use.
Diffusers Code Implementation And PR Skill
Use this skill to implement, edit, review, and prepare pull-request-ready changes for the diffusers library with high compliance to diffusers conventions.
When To Use
- Adding or editing diffusers models, pipelines, schedulers, or loaders
- Fixing bugs in inference code paths in diffusers-compatible style
- Refactoring existing diffusers code while preserving behavior
- Adding tests and docs for diffusers changes
- Preparing a PR that targets the diffusers repository
Guidance
- Consult
.github/instructions/core.instructions.mdfor relevant core runtime and diffusers integration guidance before proceeding.
Primary Objectives
- Keep behavior explicit, minimal, and inference-focused.
- Match existing diffusers architecture and code patterns.
- Preserve numerical behavior unless a behavior change is explicitly documented in the task requirements.
- Produce change sets that are clean, reviewable, and PR-ready.
Hard Rules
- Keep logic simple and readable in the main forward or call path.
- Avoid defensive, speculative, or fallback code paths unless required by existing diffusers APIs.
- Do not silently guess intent. For unsupported inputs, raise concise errors.
- Do not introduce new mandatory dependencies without maintainer agreement.
- If optional dependencies are needed, guard imports and provide proper dummy paths.
- Keep implementation torch.compile-friendly: avoid graph-break patterns in core model paths.
- Prefer native PyTorch tensor ops over external reshape helpers.
Code Structure Rules
Apply these grouped checks in priority order:
- Model-level structure and forward-path clarity.
- Attention and processor integration consistency.
- Pipeline runtime behavior and inference API expectations.
- Scheduler config and mixin conformance.
Models
- Use ModelMixin patterns and register constructor args with register_to_config.
- Keep layer invocation visible in forward, avoid hiding key module calls in extra helpers.
- Avoid hardcoded dtypes in forward paths; infer from tensors or module dtype.
- Follow existing model family patterns in src/diffusers/models/transformers.
Attention
- Keep Attention class and processor together in the model file when following the standard diffusers pattern.
- Processor should perform the compute path and use dispatch_attention_fn pattern where applicable.
- Ensure processor registration and available processor declarations are complete.
Pipelines
- Inherit from DiffusionPipeline.
- Decorate inference call with @torch.no_grad().
- Support generator for reproducibility when workflow requires it.
- Support output_type="latent" where latent output skip is expected.
- Use self.progress_bar(timesteps) in denoising loops.
- Do not build variant behavior by subclassing an unrelated existing pipeline class.
Schedulers
- Use SchedulerMixin and ConfigMixin.
- Keep scheduler config semantics consistent with existing scheduler implementations.
Import And Registration Rules
- Register new classes in relevant init.py lazy import structures.
- Ensure import structure entries are complete for all newly exposed objects.
- Validate that public imports from diffusers work after edits.
Copied Code Rules
- Respect # Copied from linkage.
- Do not manually diverge copied blocks unless intentionally breaking linkage.
- Run make fix-copies after changes that touch copied sources or copied blocks.
Change Workflow
- Gather context
- Confirm target files, model family, and expected behavior.
- Obtain reference implementation and runnable inference flow when porting.
- Plan minimal scope
- Separate structural adaptation from algorithmic changes.
- Keep one coherent workflow per change set.
- Implement
- Edit only required files.
- Preserve naming, config shape, and API contracts unless change requires otherwise.
- Validate
- Run focused tests first, then broader checks as needed.
- Confirm imports and serialization/deserialization behavior.
- Polish
- Run make style.
- Run make fix-copies.
- Re-run impacted tests.
Testing Expectations
Include tests for the exact behavior being changed:
- Model tests for shape, dtype/device behavior, serialization, and config parity
- Pipeline tests for deterministic generation paths, outputs, and parameter handling
- Scheduler tests when scheduler logic or config behavior changes
- Regression tests for any bug fix
When parity with a reference implementation is required:
- Add component-level parity checks
- Add end-to-end parity checks
- Use explicit tolerances and deterministic seeds
PR Preparation For Diffusers
When asked to prepare a PR targeting diffusers, produce:
- Scope statement
- One-paragraph summary: problem, solution, and non-goals.
- Change map
- File-by-file list describing what changed and why.
- Validation evidence
- Commands run, tests passed, and any skipped tests with reasons.
- Compatibility notes
- Backward compatibility, serialization impact, and optional dependency impact.
- Reviewer guidance
- Key files to review first, known tradeoffs, and follow-up items.
PR Quality Checklist
- Minimal focused diff
- No unrelated refactors mixed with behavior changes
- New/updated tests for changed behavior
- Docs updated when public APIs or user-facing behavior changed
- make style completed
- make fix-copies completed
- Relevant test suites pass
- Commit messages are clear and scoped
Common Failure Modes To Prevent
- Missing lazy import registration causes runtime ImportError
- New config params not registered, causing from_pretrained mismatch
- Pipeline call missing @torch.no_grad(), causing memory growth
- Hardcoded dtype assumptions break mixed precision usage
- Hidden behavior changes introduced during structural refactor
- Unnecessary dependency additions for simple tensor reshaping
Output Contract For This Skill
When using this skill, provide:
- Implementation summary
- Exact files changed
- Validation summary with command outcomes
- Residual risks or deferred follow-ups
- PR-ready summary text when requested