nemo-curator-docs

NeMo Curator Docs Maintenance

Unified skill for adding, updating, moving, and removing pages on the NeMo Curator Fern documentation site.

Scope Rule

ALL docs edits happen under fern/. The legacy docs/ directory is deprecated — do not add or move content into it. Release notes, migration guides, and every new page belong under fern/.

Layout at a Glance

fern/
├── fern.config.json          # Minimal Fern config (org + CLI version)
├── docs.yml                  # Site config: versions, tabs, redirects, libraries
├── versions/
│   ├── latest.yml            # Symlink → v26.04.yml (do not edit directly)
│   ├── v26.04.yml            # Nav tree for current train
│   ├── v26.04/pages/         # MDX content for current train
│   ├── v25.09.yml
│   └── v25.09/pages/
├── components/               # Custom TSX components (footer, etc.)
├── assets/                   # Images, SVGs, favicon
├── substitute_variables.py   # CI: resolves {{ variables }} in MDX
└── AUTODOCS_GUIDE.md         # Library reference generation guide

Current train: v26.04. Default all new pages there unless the user specifies a version.

File system                              Published URL
───────────────────────────────────────  ────────────────────────────────────────
fern/versions/v26.04/pages/              docs.nvidia.com/nemo/curator/latest/
  └─ get-started/text.mdx                  └─ get-started/text
fern/versions/v26.04.yml ── nav for ──┐  docs.nvidia.com/nemo/curator/v26.04/
fern/versions/latest.yml ─ symlink ───┘    └─ get-started/text
fern/versions/v25.09/pages/              docs.nvidia.com/nemo/curator/v25.09/
  └─ get-started/text.mdx                  └─ get-started/text

Operations

Add a Page

1. Gather: page title, target section, filename (kebab-case .mdx), subdirectory under fern/versions/v26.04/pages/.
2. Create fern/versions/v26.04/pages/<subdirectory>/<filename>.mdx:

---
description: "One-line SEO description"
categories: ["<category>"]
tags: ["<tag-1>", "<tag-2>"]
personas: ["<persona>"]
difficulty: "beginner"      # beginner | intermediate | advanced
content_type: "tutorial"     # tutorial | how-to | reference | concept | index
modality: "text-only"        # text-only | image-only | video-only | audio-only | universal
---

# <Page Title>

<content>

3. Add a nav entry in fern/versions/v26.04.yml under the correct section:

- page: <Page Title>
  path: ./v26.04/pages/<subdirectory>/<filename>.mdx
  slug: <filename>

4. If this also applies to latest, no action needed — latest.yml is a symlink to v26.04.yml.

Update a Page

1. Locate by path, title, or keyword (grep -rn in fern/versions/v26.04/pages/).
2. Content only — edit the MDX directly.
3. Title change — update the frontmatter and the - page: name in fern/versions/v26.04.yml.
4. Section move — git mv the file, update its path: in the nav, and fix all incoming links.
5. Slug change — update slug: in the nav and add a redirect in fern/docs.yml so old URLs keep working.

Remove a Page

1. Find incoming links: grep -r "<filename>" fern/versions/v26.04/pages/ --include="*.mdx".
2. git rm fern/versions/v26.04/pages/<subdirectory>/<filename>.mdx.
3. Remove the - page: block from fern/versions/v26.04.yml. If it was the last page in a section, remove the - section: block.
4. Fix or remove all incoming links found in step 1.
5. Add a redirect in fern/docs.yml if the URL was public.

Back-port to an Older Version

Only when explicitly asked. Repeat the operation in the corresponding fern/versions/vXX.YY/ tree and vXX.YY.yml nav. MDX content often diverges between trains — do not blindly copy.

Worked Example: Adding a Page

Request: "Add a how-to for benchmarking text pipelines under Curate Text."

1. Create fern/versions/v26.04/pages/curate-text/benchmarking.mdx:

   ---
   description: "Benchmark text curation pipelines and interpret throughput and memory metrics"
   categories: ["how-to"]
   tags: ["text-curation", "benchmarking", "performance"]
   personas: ["mle-focused"]
   difficulty: "intermediate"
   content_type: "how-to"
   modality: "text-only"
   ---
   
   # Benchmark Text Pipelines
   
   <content>
   

2. Add nav entry in fern/versions/v26.04.yml under the existing Curate Text section:

   - page: Benchmark Text Pipelines
     path: ./v26.04/pages/curate-text/benchmarking.mdx
     slug: benchmarking
   

3. cd fern && fern check then fern docs dev and verify the page renders at /curate-text/benchmarking.

Worked Example: Renaming a Slug (with Redirect)

Request: "Rename /curate-text/benchmarking to /curate-text/performance."

1. Update slug: in fern/versions/v26.04.yml: slug: performance.

2. (Optional) git mv the MDX file if you want the filename to match the slug.

3. Add a redirect to fern/docs.yml so old links keep working:

   redirects:
     - source: "/nemo/curator/latest/curate-text/benchmarking"
       destination: "/nemo/curator/latest/curate-text/performance"
     - source: "/nemo/curator/v26.04/curate-text/benchmarking"
       destination: "/nemo/curator/v26.04/curate-text/performance"
   

4. grep -rn "/curate-text/benchmarking" fern/versions/v26.04/pages/ and update any incoming links.

---

Content Guidelines

NeMo Curator uses Fern-native MDX components directly (unlike Dynamo, which converts GitHub callouts in CI). Do not use > [!NOTE] syntax — it will not render.

Images live in fern/assets/ (shared) or fern/versions/vXX.YY/pages/_images/ (version-scoped). Reference with root-relative paths.

Component examples:

<Tip>
If `uv` is not installed, see the [Installation Guide](/admin/installation).
</Tip>

<Warning>
GPU-accelerated dedup requires CUDA {{ recommended_cuda }} or later.
</Warning>

<Cards>
  <Card title="Text Curation" href="/get-started/text">
    Set up and run text curation workflows.
  </Card>
  <Card title="Image Curation" href="/get-started/image">
    Set up and run image curation workflows.
  </Card>
</Cards>

Frontmatter Fields

Required: description. Optional but strongly preferred: categories, tags, personas, difficulty, content_type, modality. Existing pages in the same section are the best reference for valid values.

title is taken from the - page: entry in the nav file; the MDX file itself uses an # H1 heading matching the page name.

Variable Substitution

Tokens like {{ product_name }}, {{ container_version }}, {{ current_release }}, {{ github_repo }}, {{ min_python_version }} are resolved by fern/substitute_variables.py at CI time. Use them instead of hard-coding versions or URLs. Canonical list in DEFAULT_VARIABLES at the top of that file.

Example in MDX:

Install {{ product_name }} {{ current_release }} from {{ github_repo }}.
Requires Python {{ min_python_version }}+ and CUDA {{ recommended_cuda }}.

After substitution at CI time:

Install NeMo Curator 25.09 from https://github.com/NVIDIA-NeMo/Curator.
Requires Python 3.10+ and CUDA 12.0+.

To preview substitution locally:

python fern/substitute_variables.py versions/v26.04 --version 26.04 --dry-run

Validate

cd fern
fern check                   # YAML + frontmatter validation
fern docs broken-links       # link check
fern docs dev                # localhost:3000 hot-reload preview

fern check must pass before commit. Broken-link check can be deferred but must pass in CI.

Commit & Preview

git add fern/
git commit -s -m "docs: <add|update|remove> <page-title>"

DCO sign-off (-s) is required on every commit. CI enforces it. If you forget, amend with git commit --amend --no-edit -s and force-push the branch.

PRs that touch fern/** get an automatic Fern preview URL posted as a comment by .github/workflows/fern-docs-preview-comment.yml. No manual step needed.

                    ┌─ fern-docs-ci.yml         → fern check + autodocs
PR (touches fern/) ─┼─ fern-docs-preview.yml    → preview build
                    └─ fern-docs-preview-*.yml  → 🌿 preview URL comment

Merge to main      → NO publish. Site is unchanged.

Tag push (docs/v*) → publish-fern-docs.yml      → docs.nvidia.com/nemo/curator

Publishing to Production

Merging to main does NOT publish. Production only updates when a tag matching docs/v* is pushed (or the workflow is manually dispatched from the Actions tab). Do not push tags unless the user asks.

Tag must be docs/v<MAJOR>.<MINOR>.<PATCH> — the docs/v prefix is required by the workflow trigger and the semver suffix should match the docs release in CHANGELOG.md.

# Correct — triggers publish
git tag docs/v1.1.0
git push origin docs/v1.1.0

git tag docs/v1.2.0-rc1     # pre-release suffix is fine, still matches docs/v*
git push origin docs/v1.2.0-rc1

# Wrong — these will NOT trigger publish
git tag v1.1.0              # missing docs/ prefix
git tag docs/1.1.0          # missing v
git tag docs-v1.1.0         # wrong separator

URL → version mapping after publish:

docs.nvidia.com/nemo/curator/latest/...   → symlink to current train (v26.04 today)
docs.nvidia.com/nemo/curator/v26.04/...   → 26.04 train
docs.nvidia.com/nemo/curator/v26.02/...   → 26.02 train
docs.nvidia.com/nemo/curator/v25.09/...   → 25.09 train

Version Ship Checklist (when cutting a new train)

When the user ships a new version (e.g. cutting v26.06 while v26.04 is current):

1. Copy fern/versions/v26.04/pages/ → fern/versions/v26.06/pages/ and edit content.
2. Copy fern/versions/v26.04.yml → fern/versions/v26.06.yml and update all ./v26.04/ path prefixes to ./v26.06/.
3. Repoint the symlink: ln -sf v26.06.yml fern/versions/latest.yml.
4. Update fern/docs.yml versions: list — add the new display-name, mark older trains stable.
5. Add redirect rules in fern/docs.yml for /nemo/curator/26.06/:path* → /nemo/curator/v26.06/:path* (see existing patterns).
6. Add */index.html redirect for the new version (e.g. /nemo/curator/v26.06/index.html → /nemo/curator/v26.06). The :path* rule does not match the empty-path case, so each version-root index.html needs its own explicit rule.
7. Align display-name strings with CHANGELOG.md and nemo_curator/package_info.py.

Holding a Version Back from Publish

A version is included in the published site only when it appears in the versions: block of fern/docs.yml. The MDX tree (fern/versions/vXX.YY/) and nav file (fern/versions/vXX.YY.yml) can sit in the repo unpublished — Fern doesn't auto-publish every YAML it finds.

To stage a version without publishing it (e.g. work-in-progress on v26.06 while v26.04 is current):

# fern/docs.yml
versions:
  - display-name: "Latest · v1.1.2 (26.04)"
    path: versions/latest.yml
    slug: latest
  - display-name: "26.04 · v1.1.2"
    path: versions/v26.04.yml
    slug: v26.04
  # v26.06 staged in repo but not listed here → not published

To pull an already-shipping version back (e.g. hold v26.04 while pushing fixes to older trains):

1. Remove the v26.04 entry from versions: in fern/docs.yml.
2. If latest should also stop serving 26.04 content, repoint the symlink: ln -sf v26.02.yml fern/versions/latest.yml. Otherwise leave latest alone — it will keep serving v26.04 content under /latest/ even with v26.04 removed (since latest.yml references the v26.04 pages directly).
3. Tag and push docs/v* to publish.
4. Restore the entry (and symlink) when ready.

This is a temporary maneuver — track the change so it gets reverted.

Audiences (alternative): Fern supports audiences: on versions plus separate instances: (e.g. staging vs production). This is heavier setup — only adopt if multi-instance publishing is genuinely needed. NeMo Curator does not currently configure instances. References: Fern versioning, Fern publishing.

Do not use hidden: true to hide a version from publish. Hidden versions are removed from navigation/search/indexing but remain accessible by direct URL — still effectively published.

Library Reference (Autodocs) and the Fern Cross-Ref Bug

fern/docs.yml declares one libraries: entry, and a script provides a second, temporary one (both write to fern/product-docs/nemo-curator/Full-Library-Reference/, gitignored):

• nemo-curator (input.git, committed in docs.yml) — used by CI/publish/preview; requires fern login or FERN_TOKEN. Workflows run fern docs md generate --library nemo-curator.
• nemo-curator-local (input.path: ../nemo_curator, injected temporarily by fern/scripts/generate-library-local.sh) — local dev without Fern auth; requires Docker. Run npm run generate:library:local from fern/. The entry must NOT be committed to docs.yml: fern docs dev rejects path-input libraries ("'path' input which is not yet supported") and renders a blank page, while --local generation only accepts path inputs.

For a full API reference locally (424 pages), use npm run generate:library. The local Docker parser is beta and currently produces fewer pages — fine for testing the no-auth flow, not a substitute for CI generation.

Known bug in the Fern Python library generator (filed upstream): the generator emits cross-references that miss the /nemo/curator site basepath (links use /nemo-curator/... instead of /nemo/curator/nemo-curator/...) and tacks on Sphinx-style #nemo_curator-… fragments that don't match any rendered anchor. Result: ~540 broken links across the generated API reference.

No in-repo workaround currently. A post-generation rewrite (walking the generated MDX, fixing the basepath, dropping stale fragments) is feasible but not yet wired up. Track the upstream Fern fix; revisit if it doesn't land soon.

fern/_fix_broken_links.py separately rewrites a long list of legacy URL patterns (/api/reference/api-reference/, old Sphinx slugs, etc.) on the committed MDX under fern/versions/v25.09/pages/ and fern/versions/v26.02/pages/. CI does not run it, so committed pages can drift. Re-run locally and commit the diff if you see drift:

python3 fern/_fix_broken_links.py

Redirect Quirks

• :path* does not match the empty-path case. /nemo/curator/:path*/index.html will not catch /nemo/curator/index.html — that needs its own explicit rule. Same for every version-root: /nemo/curator/{latest,vXX.YY}/index.html each need a dedicated entry. Pattern: define the explicit empty-path rules before the :path* rule.
• Order matters. Fern processes redirects top-down, first match wins. Put more specific rules above catch-alls.
• Version slugs in fern/docs.yml use the vXX.YY form (e.g. v26.04). Calendar-train forms (26.04) need redirects to the v-prefixed slug.

Debugging

Key References

---