# Debug Diff

> Debug Diff is an Agent Skill for testing & quality published in the afnanenayet/diffsitter repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Diagnose unexpected diff output by tracing the pipeline from parsing through AST processing to hunk generation. Use when diffsitter produces wrong or surprising results.

- Skill name: `debug-diff`
- Category: [Testing & Quality](https://www.openparable.com/skill/testing-quality/1)
- Source: https://github.com/afnanenayet/diffsitter/blob/HEAD/.claude/skills/debug-diff/SKILL.md
- Repository: afnanenayet/diffsitter (2,389 GitHub stars when collected)

## SKILL.md

# Debugging Unexpected Diff Output in diffsitter

When diffsitter produces wrong or surprising results, trace the issue through the pipeline. The diff pipeline has 7 stages, and problems can originate at any of them.

Ask the user for the following information if not already provided:
- The two files being compared (or representative snippets)
- The language/file extension
- Any custom config being used
- What output they expected vs what they got

## Pipeline Overview

The full pipeline (see `src/bin/diffsitter.rs`, function `run_diff`):

```
1. Language detection
2. Tree-sitter parse (file -> AST Tree)
3. AST leaf extraction (Tree -> Vec<VectorLeaf>)
4. Node filtering (exclude_kinds / include_kinds)
5. Grapheme splitting + whitespace stripping (VectorLeaf -> Vec<Entry>)
6. Myers diff (Vec<Entry> x 2 -> Vec<EditType>)
7. Hunk assembly (Vec<EditType> -> RichHunks)
8. Rendering (RichHunks -> terminal output)
```

## Stage-by-Stage Debugging

### Stage 1: Language Detection

**File**: `src/parse.rs`, function `resolve_language_str`

The language is resolved from the file extension via the `FILE_EXTS` phf_map, with optional user overrides from `GrammarConfig.file_associations`.

**Common issues:**
- Extension not mapped (e.g., `.jsx` maps to `"tsx"`, `.h` maps to `"c"` not `"cpp"`)
- User override in config shadowing the default

**Debug**: Check what language is resolved:
```sh
# Enable debug logging to see language resolution
RUST_LOG=debug diffsitter old_file new_file 2>&1 | grep -i "deduced language"
```

Or read `FILE_EXTS` in `src/parse.rs` to verify the extension mapping.

### Stage 2: Tree-sitter Parse

**File**: `src/parse.rs`, function `parse_file`

Creates a `Parser`, sets the language, reads the file to a string, and calls `parser.parse(&text, None)`.

**Common issues:**
- Grammar ABI version mismatch (`AbiOutOfRange` error) -- the grammar was compiled against an incompatible tree-sitter version.
- Parse failure returns `None` from `parser.parse()`, resulting in a `TSParseFailure` error.
- File encoding issues (tree-sitter expects UTF-8).

**Debug**: If parsing succeeds but the tree looks wrong, the grammar itself may have a bug for that language construct. Check tree-sitter's own playground or CLI to inspect the AST.

### Stage 3: AST Leaf Extraction

**File**: `src/input_processing.rs`, function `build`

Recursively walks the tree via `node.children(&mut cursor)`. Collects leaf nodes (nodes with `child_count() == 0`) OR nodes whose `kind()` matches a `pseudo_leaf_types` entry.

**Key behaviors:**
- Empty byte ranges are skipped (`node.byte_range().is_empty()`)
- Nodes that are pure newlines (after removing `\n`, `\r`, `\r\n`) are skipped -- this is a workaround for the Go parser
- Pseudo-leaf types (configured in `input_processing.pseudo_leaf_types`) treat certain non-leaf nodes as leaves. Default: `{"markdown": {"inline"}}`. This is critical for text-heavy documents.

**Common issues:**
- Missing diffs in markdown/prose: Check if `pseudo_leaf_types` includes the right node types for that language. Without `"inline"` for markdown, large text blocks are treated as single atoms.
- Unexpected nodes included: The grammar may expose more leaf nodes than expected (e.g., punctuation, delimiters).

### Stage 4: Node Filtering

**File**: `src/input_processing.rs`, method `TreeSitterProcessor::should_include_node`

Filters nodes based on `exclude_kinds` and `include_kinds` from the config:
- `exclude_kinds` takes precedence: if a node's `kind()` is in this set, it is excluded.
- `include_kinds`: if set, only nodes whose `kind()` is in this set are included (unless also excluded).
- If neither is set, all nodes pass through.

**Common issues:**
- User config has `exclude_kinds` or `include_kinds` that filters out relevant nodes.
- The node `kind()` string doesn't match what the user expects (tree-sitter kind names are grammar-specific).

**Debug**: Check what node kinds exist for a language using tree-sitter's node types. The `kind()` strings come from the grammar definition.

### Stage 5: Grapheme Splitting and Whitespace Stripping

**File**: `src/input_processing.rs`, method `VectorLeaf::split_on_graphemes`

If `split_graphemes` is enabled (default: true), each leaf's text is split into individual Unicode graphemes, each becoming its own `Entry` with precise row/column positions. If `strip_whitespace` is enabled (default: true), whitespace-only graphemes are skipped.

**Common issues:**
- **Whitespace-only diffs not showing**: `strip_whitespace: true` (the default) means pure whitespace/indentation changes are invisible. This is by design for AST-based diffing. If the user wants to see whitespace changes, they need `"strip-whitespace": false` in their config.
- **Performance with large files**: `split_graphemes: true` generates many entries for large text nodes. Setting `"split-graphemes": false` trades granularity for speed.
- **Line position bugs**: The grapheme splitter tracks row/column positions. If `line_offset == 0`, it offsets from the node's `start_position().column`. Otherwise it resets the column to `idx` (the byte offset within the line). Bugs here manifest as incorrect column highlighting in the rendered output.

### Stage 6: Myers Diff

**File**: `src/diff.rs`, struct `Myers`, method `diff`

Implements the classic Myers "An O(ND) Difference Algorithm" with divide-and-conquer via middle snake. Takes two `&[Entry]` slices, produces `Vec<EditType<&Entry>>` where `EditType` is either `Addition` or `Deletion`.

**Key optimizations:**
- Common prefix/suffix are skipped before running the main algorithm (via `common_prefix_len` / `common_suffix_len` which use `get_unchecked` for speed).
- `Entry` equality is based on `kind_id` AND `text` (see the `PartialEq` impl) -- two nodes are equal only if they have the same tree-sitter kind AND identical text content.

**Common issues:**
- **Too many diffs reported**: If entries that should be equal are not, check if `kind_id` differs between them. Two nodes with the same text but different grammar kinds (e.g., `identifier` vs `type_identifier` in Rust) are considered different.
- **No diffs reported when expected**: If entries that should differ are comparing as equal, verify the text content after whitespace stripping.

### Stage 7: Hunk Assembly

**File**: `src/diff.rs`, struct `RichHunksBuilder`

Converts the flat edit script into grouped hunks. Each `EditType::Addition` becomes `DocumentType::New`, each `EditType::Deletion` becomes `DocumentType::Old`. Consecutive edits on adjacent lines are grouped into the same `Hunk`. Non-adjacent edits start a new hunk.

**Common issues:**
- **`PriorLine` or `PriorColumn` errors**: These indicate the edit script produced entries in non-ascending order, which is a bug in the diff or input processing stage.
- **Hunks splitting unexpectedly**: If edits on adjacent lines end up in separate hunks, there may be a gap in line numbers caused by filtered-out nodes.

### Stage 8: Rendering

**File**: `src/render/unified.rs` (for unified renderer), `src/render/json.rs` (for JSON)

**Common issues with unified renderer:**
- **Line index out of bounds**: The `print_hunk` method accesses `lines[line_index]`. If `line_index >= lines.len()`, it logs an error and skips the line (in release) or panics via `debug_assert!` (in debug).
- **Column range panics**: `print_line` indexes into the line text using byte column ranges from entries. If the entry positions don't align with the actual text bytes, this can panic with a slice bounds error.
- **Missing terminal colors**: If output is piped, `term_info` may be `None`. The unified renderer still works but won't have terminal width info for title formatting.

## Quick Diagnostic Commands

```sh
# Run with full debug logging
RUST_LOG=trace diffsitter old_file new_file 2>debug.log

# Output as JSON to inspect raw diff data
diffsitter --renderer json old_file new_file | jq .

# Force a specific language
diffsitter --file-type python old_file new_file

# Run without config to eliminate config issues
diffsitter --no-config old_file new_file
```

## Common Failure Modes Summary

| Symptom | Likely Stage | Check |
|---------|-------------|-------|
| "Unsupported extension" error | 1 (Language detection) | Extension in `FILE_EXTS`? |
| Parse error / empty AST | 2 (Tree-sitter parse) | Grammar ABI compatible? File is valid UTF-8? |
| No diff when content clearly changed | 5 (Whitespace stripping) | `strip_whitespace` removing the changes? |
| Missing diffs in markdown/prose | 3 (Leaf extraction) | `pseudo_leaf_types` configured for language? |
| Diffs include too many trivial nodes | 4 (Node filtering) | Set `exclude_kinds` for noise (e.g., `"comment"`, `"string"`) |
| Wrong columns highlighted | 5 (Grapheme splitting) | Grapheme position calculation bug |
| Panic on "index out of bounds" | 7/8 (Hunk assembly / Rendering) | Line index mismatch between entry positions and actual text |
| Extremely slow on large files | 5/6 (Graphemes / Myers) | Try `"split-graphemes": false` or check if diff is O(ND) worst-case |

---

Source: https://www.openparable.com/skill/testing-quality/debug-diff (OpenParable). Cite the canonical URL above.
