paper-notes

Paper Notes

Produce consistent, searchable paper notes that later steps (claims, visuals, writing) can reliably synthesize.

This is still NO PROSE: keep notes as bullets / short fields, not narrative paragraphs.

Load Order

Always read:

• references/overview.md
• references/note_schema.md

Read by task:

• references/limitation_taxonomy.md when writing or reviewing limitations (avoid boilerplate)
• references/result_extraction_examples.md when extracting key_results (good vs bad examples)
• references/source_text_hygiene.md when result/limitation fields still preserve paper self-narration or author-result wrappers

Machine-readable assets:

• assets/note_schema.json — JSONL record schema for validation
• assets/evidence_tags.json — evidence bank tagging categories (extensible without code changes)
• assets/source_text_hygiene.json — note-field source sentence cleanup policy

Script Boundary

Use scripts/run.py only for:

• deterministic scaffold generation from core_set + metadata
• priority selection based on mapping coverage
• evidence bank construction from structured note fields

Do not treat run.py as the place for:

• paper-specific limitation prose (use references/limitation_taxonomy.md for guidance)
• domain-specific evaluation heuristics hidden in code
• reader-facing narrative text

Role cards (prompt-level guidance)

• Close Reader

  • Mission: extract what is specific and checkable (setup, method, metrics, limits).
  • Do: name concrete tasks/benchmarks and what the paper actually measures.
  • Avoid: generic summary boilerplate that could fit any paper.

• Results Recorder

  • Mission: capture evaluation anchors that later writing needs.
  • Do: record task + metric + constraints (budget/tool access) whenever available.
  • Avoid: copying numbers without the evaluation setting that makes them meaningful.
  • Avoid: promoting artifact introductions (X enables ..., our framework features ...) into key_results.
  • Avoid: promoting benchmark-positioning, field-motivation, or author-navigation lines (we apply ... and show ..., we then discuss how ...) into key_results.

• Limitation Logger

  • Mission: capture the caveats that change interpretation.
  • Do: write paper-specific limitations (protocol mismatch, missing ablations, threat model gaps).
  • Avoid: repeated generic limitations like “may not generalize” without specifics.

When to use

• After you have a core set (and ideally a mapping) and need evidence-ready notes.
• Before writing a survey draft.

Inputs

• papers/core_set.csv
• Optional: outline/mapping.tsv (to prioritize)
• Optional: papers/fulltext_index.jsonl + papers/fulltext/*.txt (if running in fulltext mode)

Outputs

• papers/paper_notes.jsonl (JSONL; one record per paper)
• papers/evidence_bank.jsonl (JSONL; addressable evidence snippets derived from notes; profile target: course paper >=4, A150++ >=7 items/paper on average)

Decision: evidence depth

• If you have extracted text (papers/fulltext/*.txt) → enrich key papers using fulltext snippets and set evidence_level: "fulltext".
• If you only have abstracts (default) → keep long-tail notes abstract-level, but still fully enrich high-priority papers (see below).

Workflow (heuristic)

Uses: outline/mapping.tsv, papers/fulltext_index.jsonl.

1. Ensure coverage: every paper_id in papers/core_set.csv must have one JSONL record.
2. Use mapping to choose high-priority papers:
   • heavily reused across subsections
   • pinned classics (ReAct/Toolformer/Reflexion… if in scope)
3. For high-priority papers, capture:
   • 3–6 summary bullets (what’s new, what problem setting, what’s the loop)
   • method (mechanism and architecture; what differs from baselines)
   • key_results (benchmarks/metrics; include numbers if available)
   • limitations (specific assumptions/failure modes; avoid generic boilerplate)
4. For long-tail papers:
   • keep summary bullets short (abstract-derived is OK)
   • still include at least one limitation, but make it specific when possible
5. Assign a stable bibkey for each paper for citation generation.

Quality checklist

• Coverage: every paper_id in papers/core_set.csv appears in papers/paper_notes.jsonl.

• High-priority papers have non-TODO method/results/limitations.

• Limitations are not copy-pasted across many papers.

• evidence_level is set correctly (abstract vs fulltext).

• Evidence bank: papers/evidence_bank.jsonl exists and meets the selected profile (course paper >=4; A150++ >=7 items/paper on average).

Helper script (optional)

Quick Start

• uv run python .codex/skills/paper-notes/scripts/run.py --help
• uv run python .codex/skills/paper-notes/scripts/run.py --workspace <workspace>

All Options

• See --help (this helper is intentionally minimal)

Examples

• Generate notes, then optionally enrich priority=high papers:
  • Run the helper once, then refine papers/paper_notes.jsonl (e.g., add full-text details for key papers and diversify limitations).

Notes

• The helper writes deterministic metadata/abstract-level notes and marks key papers with priority=high.
• In pipeline.py --strict it will be blocked if high-priority notes are incomplete (missing method/key_results/limitations) or contain placeholders.

Troubleshooting

Common Issues

Issue: High-priority notes still look like scaffolds

Symptom:

• Quality gate reports missing method/key_results or TODO placeholders.

Causes:

• Notes were generated from abstracts only; key papers weren’t enriched.

Solutions:

• Fully enrich priority=high papers: method, ≥1 key_results, ≥3 summary_bullets, ≥1 concrete limitations.
• If you need full text evidence, run pdf-text-extractor in fulltext mode for key papers.

Issue: Repeated limitations across many papers

Symptom:

• Quality gate reports repeated limitation boilerplate.

Causes:

• Copy-pasted limitations instead of paper-specific failure modes/assumptions.

Solutions:

• Replace boilerplate with paper-specific limitations (setup, data, evaluation gaps, failure cases).

Recovery Checklist

• papers/paper_notes.jsonl covers all papers/core_set.csv paper_ids.
• ≥80% of priority=high notes satisfy method/results/limitations completeness.
• No TODO remains in high-priority notes.