# Compare Python Typecheckers

> Compare Python Typecheckers is an Agent Skill for testing & quality published in the JetBrains/intellij-community repository on GitHub: a SKILL.md instruction file an AI agent can load to do this task.

Run a Python file or snippet through the real third-party type checkers (ty, pyrefly, basedpyright, mypy, zuban) via uvx and collate their output into one Markdown report. Use when evaluating how a type should be inferred, debugging PyCharm code-insight / type-engine behaviour, or checking where the external checkers agree or disagree on a snippet.

- Skill name: `compare-python-typecheckers`
- Category: [Testing & Quality](https://www.openparable.com/skill/testing-quality/1)
- Source: https://github.com/JetBrains/intellij-community/blob/HEAD/.claude/skills/compare-python-typecheckers/SKILL.md
- Repository: JetBrains/intellij-community (20,363 GitHub stars when collected)

## SKILL.md

<!-- Generated by community/.ai/render-guides.mjs; edit community/.agents/skills/compare-python-typecheckers/SKILL.md -->

# Compare Python type checkers

When reasoning about how PyCharm *should* infer a type or whether something ought
to be a type error, cross-checking against the real third-party checkers is the
fastest way to gain insight — and to see where they disagree. This skill runs
them all on a scratch file or snippet and collates the output into a single
report.

## Run it

`${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py` invokes each checker
through `uvx` (fetched on demand — no install; the network is used on first run)
and prints a Markdown report: a summary table (exit code / verdict / timing per
tool) followed by each tool's raw output.

```bash
# Inline snippet:
uv run ${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py -c 'def f(x: int) -> int:
    return x

f("a")'

# An existing file:
uv run ${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py path/to/test.py

# A subset, written to a file:
uv run ${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py test.py --tools ty,mypy -o /tmp/report.md
```

Flags: `-c/--code` (inline snippet), `-t/--tools ty,mypy,…` (subset), `-o/--output`
(write to file), `--timeout` (per-checker seconds, default 180). The script exits
0 whenever it produced a report — findings live in the report, not the exit code.

## Reading the result

- The summary table shows each tool's exit code (**0 = nothing flagged**,
  non-zero = the checker reported an issue or failed to run), a verdict, and time.
- Output **formats differ** per tool, so read each section, not just the table:
  - `ty` (Astral) and `pyrefly` (Meta) each have their own diagnostic style.
  - `basedpyright` is pyright-based and also emits extra diagnostics such as
    `reportUnusedCallResult`.
  - `mypy` and `zuban` share the same message format (`zuban` is mypy-compatible).
- The checkers track their latest releases through `uvx`, so behaviour can shift
  over time — record the date when capturing results in a YouTrack issue or test.

## Manual one-offs

The exact per-tool invocations, if you want to run just one by hand (note
`basedpyright` takes the file positionally — no `check` subcommand):

| Tool | Command |
|------|---------|
| ty | `uvx ty check test.py` |
| pyrefly | `uvx pyrefly check test.py` |
| basedpyright | `uvx basedpyright test.py` |
| mypy | `uvx mypy test.py` |
| zuban | `uvx zuban check test.py` |

---

Source: https://www.openparable.com/skill/testing-quality/compare-python-typecheckers (OpenParable). Cite the canonical URL above.
